Jul 16 10:30:18.123199 ip-10-0-128-213 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 16 10:30:18.123212 ip-10-0-128-213 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 16 10:30:18.123221 ip-10-0-128-213 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 16 10:30:18.123698 ip-10-0-128-213 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 16 10:30:28.343268 ip-10-0-128-213 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 16 10:30:28.343288 ip-10-0-128-213 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 3cd18baba4944491841f3c657488097e --
Jul 16 10:32:25.153070 ip-10-0-128-213 systemd[1]: Starting Kubernetes Kubelet...
Jul 16 10:32:25.520033 ip-10-0-128-213 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 10:32:25.520033 ip-10-0-128-213 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 16 10:32:25.520033 ip-10-0-128-213 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 10:32:25.520033 ip-10-0-128-213 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 16 10:32:25.520033 ip-10-0-128-213 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 10:32:25.521085 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.520935    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 16 10:32:25.525025 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525010    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 10:32:25.525025 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525025    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525029    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525033    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525037    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525040    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525042    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525045    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525048    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525051    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525054    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525057    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525060    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525063    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525066    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525069    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525072    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525075    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525077    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525080    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525083    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 10:32:25.525091 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525085    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525088    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525091    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525094    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525097    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525100    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525104    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525107    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525115    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525118    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525120    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525124    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525126    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525129    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525131    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525134    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525137    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525139    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525142    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525144    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 10:32:25.525599 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525147    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525150    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525153    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525158    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525162    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525165    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525169    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525172    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525175    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525178    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525181    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525184    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525187    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525189    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525194    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525197    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525200    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525203    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525206    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 10:32:25.526126 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525209    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525212    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525215    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525217    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525220    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525223    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525226    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525232    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525235    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525239    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525242    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525244    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525248    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525250    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525253    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525256    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525259    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525262    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525264    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 10:32:25.526604 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525267    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525270    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525275    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525279    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525282    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525285    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525288    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525291    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525294    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525297    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525300    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525303    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525306    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525309    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525312    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525314    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525317    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525320    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525323    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 10:32:25.527135 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.525326    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526088    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526095    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526098    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526101    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526104    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526107    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526109    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526112    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526117    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526122    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526125    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526128    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526131    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526134    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526138    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526142    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526144    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526147    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526150    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 10:32:25.527611 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526153    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526156    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526158    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526161    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526164    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526167    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526170    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526173    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526175    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526178    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526181    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526184    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526187    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526189    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526192    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526195    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526198    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526200    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526203    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 10:32:25.528163 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526206    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526208    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526211    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526214    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526219    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526224    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526227    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526231    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526235    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526238    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526242    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526245    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526248    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526252    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526255    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526257    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526260    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526263    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526265    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 10:32:25.528646 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526268    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526271    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526273    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526276    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526279    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526282    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526285    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526288    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526290    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526293    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526296    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526298    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526301    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526304    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526306    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526310    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526313    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526316    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526319    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526321    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 10:32:25.529148 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526324    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526327    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526329    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526332    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526335    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526338    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526340    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526344    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526347    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526349    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526352    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526355    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526357    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526361    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526364    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526367    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526369    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526372    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526376    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526379    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 10:32:25.529680 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526381    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526384    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526763    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526768    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526771    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526774    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526776    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526780    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526782    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526786    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526788    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526791    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526794    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526797    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526800    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526803    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526806    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526809    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526811    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526814    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 10:32:25.530216 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526831    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526834    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526838    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526840    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526843    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526846    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526849    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526852    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526855    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526857    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526862    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526866    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526869    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526874    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526878    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526881    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526884    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526887    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526890    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 10:32:25.530733 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526893    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526896    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526899    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526902    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526905    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526908    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526910    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526913    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526915    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526919    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526922    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526925    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526928    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526930    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526933    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526936    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526939    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526941    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526944    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526947    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 10:32:25.531238 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526950    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526952    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526955    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526958    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526961    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526963    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526967    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526969    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526972    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526974    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526977    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526980    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526983    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526986    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526989    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526992    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526994    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.526997    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527000    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 10:32:25.531755 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527002    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527005    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527008    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527011    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527014    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527017    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527019    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527022    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527025    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527027    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527031    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527033    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527037    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527040    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527043    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527045    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527048    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527051    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527054    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527056    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 10:32:25.532294 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527059    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527062    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527064    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527137    2576 flags.go:64] FLAG: --address="0.0.0.0"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527145    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527154    2576 flags.go:64] FLAG: --anonymous-auth="true"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527159    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527176    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527180    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527184    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527189    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527192    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527195    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527199    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527203    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527206    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527210    2576 flags.go:64] FLAG: --cgroup-root=""
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527214    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527217    2576 flags.go:64] FLAG: --client-ca-file=""
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527220    2576 flags.go:64] FLAG: --cloud-provider="external"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527223    2576 flags.go:64] FLAG: --cluster-dns="[]"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527228    2576 flags.go:64] FLAG: --cluster-domain=""
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527231    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527234    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 16 10:32:25.532810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527238    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527241    2576 flags.go:64] FLAG: --container-log-max-files="5"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527245    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527248    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527251    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527255    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527258    2576 flags.go:64] FLAG: --contention-profiling="false"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527261    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527264    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527267    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527271    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527275    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527279    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527284    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527287    2576 flags.go:64] FLAG: --enable-load-reader="false"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527290    2576 flags.go:64] FLAG: --enable-server="true"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527295    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527299    2576 flags.go:64] FLAG: --event-burst="100"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527302    2576 flags.go:64] FLAG: --event-qps="50"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527306    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527309    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527312    2576 flags.go:64] FLAG: --eviction-hard=""
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527316    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527319    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527322    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 16 10:32:25.533452 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527326    2576 flags.go:64] FLAG: --eviction-soft=""
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527330    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527333    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527336    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527339    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527342    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527345    2576 flags.go:64] FLAG: --fail-swap-on="true"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527348    2576 flags.go:64] FLAG: --feature-gates=""
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527351    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527355    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527358    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527361    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527364    2576 flags.go:64] FLAG: --healthz-port="10248"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527368    2576 flags.go:64] FLAG: --help="false"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527371    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-128-213.ec2.internal"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527374    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527377    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527380    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527384    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527387    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527392    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527395    2576 flags.go:64] FLAG: --image-service-endpoint=""
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527398    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 16 10:32:25.534136 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527403    2576 flags.go:64] FLAG: --kube-api-burst="100"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527406    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527409    2576 flags.go:64] FLAG: --kube-api-qps="50"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527412    2576 flags.go:64] FLAG: --kube-reserved=""
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527415    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527418    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527421    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527424    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527427    2576 flags.go:64] FLAG: --lock-file=""
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527430    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527434    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527437    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527442    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527446    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527449    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527452    2576 flags.go:64] FLAG: --logging-format="text"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527455    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527458    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527461    2576 flags.go:64] FLAG: --manifest-url=""
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527464    2576 flags.go:64] FLAG: --manifest-url-header=""
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527468    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527471    2576 flags.go:64] FLAG: --max-open-files="1000000"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527476    2576 flags.go:64] FLAG: --max-pods="110"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527478    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527481    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 16 10:32:25.534766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527484    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527487    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527490    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527493    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527498    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527505    2576 flags.go:64] FLAG: --node-status-max-images="50"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527508    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527513    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527517    2576 flags.go:64] FLAG: --pod-cidr=""
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527520    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527526    2576 flags.go:64] FLAG: --pod-manifest-path=""
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527529    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527533    2576 flags.go:64] FLAG: --pods-per-core="0"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527536    2576 flags.go:64] FLAG: --port="10250"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527539    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527542    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0caf901d75a8beba6"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527545    2576 flags.go:64] FLAG: --qos-reserved=""
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527549    2576 flags.go:64] FLAG: --read-only-port="10255"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527552    2576 flags.go:64] FLAG: --register-node="true"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527555    2576 flags.go:64] FLAG: --register-with-taints=""
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527559    2576 flags.go:64] FLAG: --registry-burst="10"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527562    2576 flags.go:64] FLAG: --registry-qps="5"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527565    2576 flags.go:64] FLAG: --reserved-cpus=""
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527568    2576 flags.go:64] FLAG: --reserved-memory=""
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527572    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 16 10:32:25.535435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527575    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527578    2576 flags.go:64] FLAG: --rotate-certificates="false"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527581    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527584    2576 flags.go:64] FLAG: --runonce="false"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527587    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527591    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527593    2576 flags.go:64] FLAG: --seccomp-default="false"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527596    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527599    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527602    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527605    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527609    2576 flags.go:64] FLAG: --storage-driver-password="root"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527614    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527617    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527622    2576 flags.go:64] FLAG: --storage-driver-user="root"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527625    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527628    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527631    2576 flags.go:64] FLAG: --system-cgroups=""
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527634    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527640    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527643    2576 flags.go:64] FLAG: --tls-cert-file=""
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527646    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527651    2576 flags.go:64] FLAG: --tls-min-version=""
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527654    2576 flags.go:64] FLAG: --tls-private-key-file=""
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527657    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 16 10:32:25.536121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527660    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527663    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527667    2576 flags.go:64] FLAG: --v="2"
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527671    2576 flags.go:64] FLAG: --version="false"
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527675    2576 flags.go:64] FLAG: --vmodule=""
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527679    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.527682    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527798    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527801    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527805    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527807    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527810    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527813    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527830    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527833    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527836    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527839    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527842    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527845    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527850    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527853    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 10:32:25.536786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527857    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527860    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527863    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527866    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527868    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527871    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527874    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527877    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527880    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527882    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527885    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527887    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527890    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527894    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527896    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527899    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527902    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527904    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527907    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527910    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 10:32:25.537379 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527912    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527915    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527918    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527920    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527923    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527926    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527928    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527931    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527934    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527936    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527941    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527944    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527956    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527959    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527962    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527965    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527968    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527970    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527975    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 10:32:25.537914 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527979    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527982    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527984    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527987    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527990    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527993    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527996    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.527999    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528002    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528006    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528008    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528011    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528014    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528016    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528019    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528023    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528027    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528030    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528034    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 10:32:25.538409 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528037    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528041    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528044    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528047    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528051    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528054    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528061    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528064    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528067    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528070    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528073    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528076    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528079    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528082    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528085    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528088    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528091    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528094    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528096    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528099    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 10:32:25.538925 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528102    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528105    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528108    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528111    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528113    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528116    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.528119    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.528709    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.535721    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.535736    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.535756    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.535763    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535841    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535846    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535850    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 10:32:25.539446 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535853    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535857    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535860    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535864    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535867    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535871    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535874    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535879    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535883    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535886    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535889    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535892    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535896    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535900    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535903    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535905    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535908    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535911    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535914    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 10:32:25.539856 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535917    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535920    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535922    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535925    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535928    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535931    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535934    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535939    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535942    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535945    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535948    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535950    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535953    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535956    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535959    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535961    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535964    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535967    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535970    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 10:32:25.540352 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535975    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535978    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535982    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535985    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535988    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535991    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535994    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.535997    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536000    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536003    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536005    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536008    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536011    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536014    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536016    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536019    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536022    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536025    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536028    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536031    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 10:32:25.541020 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536034    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536037    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536040    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536043    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536045    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536048    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536051    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536054    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536057    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536059    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536062    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536065    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536067    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536070    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536072    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536075    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536078    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536081    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536083    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536086    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 10:32:25.541518 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536089    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536091    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536094    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536097    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536100    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536102    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536105    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536108    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536110    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536113    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536116    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536119    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536122    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536125    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536128    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536131    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536134    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 10:32:25.542073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536136    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.536142    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536259    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536264    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536268    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536271    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536274    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536277    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536280    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536284    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536288    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536293    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536296    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536299    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 10:32:25.542537 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536302    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536305    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536308    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536311    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536314    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536316    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536319    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536322    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536324    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536327    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536329    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536332    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536335    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536338    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536341    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536343    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536347    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536350    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536352    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536355    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 10:32:25.542920 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536358    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536361    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536363    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536366    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536369    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536372    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536375    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536377    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536380    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536383    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536385    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536388    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536391    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536393    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536396    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536399    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536401    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536404    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536407    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536409    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 10:32:25.543441 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536412    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536415    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536417    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536420    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536423    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536426    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536429    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536431    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536434    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536437    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536440    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536442    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536445    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536448    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536451    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536453    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536456    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536458    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536461    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536464    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 10:32:25.544347 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536466    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536469    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536472    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536474    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536477    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536479    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536482    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536485    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536488    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536491    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536493    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536496    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536498    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536501    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536504    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536506    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536509    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536512    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536515    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536517    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 10:32:25.545155 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536520    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 10:32:25.545727 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536522    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 10:32:25.545727 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536525    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 10:32:25.545727 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536528    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 10:32:25.545727 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536531    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 10:32:25.545727 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536534    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 10:32:25.545727 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:25.536537    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 10:32:25.545727 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.536543    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 10:32:25.545727 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.536657    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 16 10:32:25.545727 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.538722    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 16 10:32:25.545727 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.540037    2576 server.go:1019] "Starting client certificate rotation"
Jul 16 10:32:25.545727 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.540138    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 10:32:25.545727 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.540182    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 10:32:25.545727 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.542385    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 10:32:25.546140 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.545862    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 10:32:25.557925 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.557906    2576 log.go:25] "Validated CRI v1 runtime API"
Jul 16 10:32:25.560244 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.560229    2576 log.go:25] "Validated CRI v1 image API"
Jul 16 10:32:25.562202 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.562187    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 16 10:32:25.564707 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.564686    2576 fs.go:135] Filesystem UUIDs: map[7390b855-fdd5-409f-80c9-ca9e9c870b67:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 eea6003e-f903-4a68-9ba9-b95cde46ab9e:/dev/nvme0n1p3]
Jul 16 10:32:25.564770 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.564706    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 16 10:32:25.570272 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.570166    2576 manager.go:216] Machine: {Timestamp:2026-07-16 10:32:25.568679826 +0000 UTC m=+0.305715091 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099211 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2da931e88406b09bb6deeac7a0fb03 SystemUUID:ec2da931-e884-06b0-9bb6-deeac7a0fb03 BootID:3cd18bab-a494-4491-841f-3c657488097e Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:bf:4f:95:be:f5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:bf:4f:95:be:f5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:48:85:71:e4:2c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 16 10:32:25.570272 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.570267    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 16 10:32:25.570423 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.570350    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 16 10:32:25.573331 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.573300    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 16 10:32:25.573531 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.573339    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-213.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 16 10:32:25.574661 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.574648    2576 topology_manager.go:138] "Creating topology manager with none policy"
Jul 16 10:32:25.574730 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.574667    2576 container_manager_linux.go:309] "Creating device plugin manager"
Jul 16 10:32:25.574730 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.574679    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 10:32:25.575267 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.575254    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 10:32:25.576412 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.576395    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 10:32:25.576501 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.576476    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 16 10:32:25.579462 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.579449    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 10:32:25.579582 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.579572    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 16 10:32:25.579622 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.579613    2576 kubelet.go:486] "Attempting to sync node with API server"
Jul 16 10:32:25.579653 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.579630    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 16 10:32:25.579653 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.579645    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 16 10:32:25.579716 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.579655    2576 kubelet.go:398] "Adding apiserver pod source"
Jul 16 10:32:25.579716 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.579664    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 16 10:32:25.580053 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.580040    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 10:32:25.580123 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.580058    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 10:32:25.582477 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.582460    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 16 10:32:25.584170 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584153    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 16 10:32:25.584249 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584194    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 16 10:32:25.584474 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584459    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 16 10:32:25.584550 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584483    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 16 10:32:25.584550 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584492    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 16 10:32:25.584550 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584502    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 16 10:32:25.584550 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584514    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 16 10:32:25.584550 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584523    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 16 10:32:25.584912 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584900    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 16 10:32:25.584976 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584914    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 16 10:32:25.584976 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584925    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 16 10:32:25.584976 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584934    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 16 10:32:25.584976 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584947    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 16 10:32:25.584976 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.584961    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 16 10:32:25.585702 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.585690    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 16 10:32:25.585763 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.585704    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 16 10:32:25.591024 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.591008    2576 server.go:1268] "Started kubelet"
Jul 16 10:32:25.591523 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.591409    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 16 10:32:25.591523 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.591479    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 16 10:32:25.591688 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.591632    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 16 10:32:25.591688 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.591650    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 16 10:32:25.591801 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.591700    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 16 10:32:25.591801 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.591783    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 16 10:32:25.592310 ip-10-0-128-213 systemd[1]: Started Kubernetes Kubelet.
Jul 16 10:32:25.595302 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.595276    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 16 10:32:25.595837 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.595823    2576 server.go:310] "Adding debug handlers to kubelet server"
Jul 16 10:32:25.597234 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.597216    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 16 10:32:25.599620 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.599601    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 16 10:32:25.599620 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.599610    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 16 10:32:25.599973 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.599774    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found"
Jul 16 10:32:25.599973 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.598959    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-213.ec2.internal.18c2be9413ee8035  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-213.ec2.internal,UID:ip-10-0-128-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-213.ec2.internal,},FirstTimestamp:2026-07-16 10:32:25.590972469 +0000 UTC m=+0.328007734,LastTimestamp:2026-07-16 10:32:25.590972469 +0000 UTC m=+0.328007734,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-213.ec2.internal,}"
Jul 16 10:32:25.600244 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.600224    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 16 10:32:25.600313 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.600227    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 16 10:32:25.600313 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.600263    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 16 10:32:25.600415 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.600355    2576 factory.go:55] Registering systemd factory
Jul 16 10:32:25.600415 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.600383    2576 factory.go:223] Registration of the systemd container factory successfully
Jul 16 10:32:25.600513 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.600361    2576 reconstruct.go:97] "Volume reconstruction finished"
Jul 16 10:32:25.600513 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.600431    2576 reconciler.go:29] "Reconciler: start to sync state"
Jul 16 10:32:25.600644 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.600630    2576 factory.go:153] Registering CRI-O factory
Jul 16 10:32:25.600689 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.600650    2576 factory.go:223] Registration of the crio container factory successfully
Jul 16 10:32:25.600734 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.600696    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 16 10:32:25.600734 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.600717    2576 factory.go:103] Registering Raw factory
Jul 16 10:32:25.600734 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.600735    2576 manager.go:1196] Started watching for new ooms in manager
Jul 16 10:32:25.601349 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.601333    2576 manager.go:349] Starting recovery of all containers
Jul 16 10:32:25.601924 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.601896    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lsngp"
Jul 16 10:32:25.611604 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.611422    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 16 10:32:25.611730 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.611706    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 16 10:32:25.612352 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.612329    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lsngp"
Jul 16 10:32:25.612964 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.612948    2576 manager.go:354] Recovery completed
Jul 16 10:32:25.619657 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.619642    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 10:32:25.621492 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.621476    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 10:32:25.621557 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.621507    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 10:32:25.621557 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.621520    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID"
Jul 16 10:32:25.622190 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.622171    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 16 10:32:25.622190 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.622189    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 16 10:32:25.622287 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.622204    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 10:32:25.622287 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.622216    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 10:32:25.622959 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.622942    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 10:32:25.623013 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.622973    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 10:32:25.623013 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.622986    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID"
Jul 16 10:32:25.623140 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.623127    2576 policy_none.go:49] "None policy: Start"
Jul 16 10:32:25.623189 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.623145    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 16 10:32:25.623189 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.623157    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 16 10:32:25.624189 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.624178    2576 policy_none.go:47] "Start"
Jul 16 10:32:25.662428 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.662411    2576 manager.go:337] "Starting Device Plugin manager"
Jul 16 10:32:25.662519 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.662446    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 16 10:32:25.662519 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.662457    2576 server.go:87] "Starting device plugin registration server"
Jul 16 10:32:25.662758 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.662729    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 16 10:32:25.662758 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.662743    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 16 10:32:25.662900 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.662886    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 16 10:32:25.663375 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.662975    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 16 10:32:25.663375 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.662990    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 16 10:32:25.663516 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.663439    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 16 10:32:25.663516 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.663487    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-213.ec2.internal\" not found"
Jul 16 10:32:25.767064 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.767026    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 10:32:25.768359 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.768341    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 10:32:25.768442 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.768373    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 10:32:25.768442 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.768386    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID"
Jul 16 10:32:25.768442 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.768419    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-213.ec2.internal"
Jul 16 10:32:25.777546 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.777529    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-213.ec2.internal"
Jul 16 10:32:25.777637 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.777554    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-213.ec2.internal\": node \"ip-10-0-128-213.ec2.internal\" not found"
Jul 16 10:32:25.793330 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.793305    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 16 10:32:25.794611 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.794591    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 16 10:32:25.794699 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.794629    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 16 10:32:25.794699 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.794649    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 16 10:32:25.794699 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.794680    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 16 10:32:25.797036 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.797013    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 10:32:25.799243 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.799227    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found"
Jul 16 10:32:25.895721 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.895663    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal"]
Jul 16 10:32:25.895922 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.895811    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 10:32:25.897629 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.897609    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 10:32:25.897740 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.897641    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 10:32:25.897740 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.897652    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID"
Jul 16 10:32:25.899060 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.899045    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 10:32:25.899202 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.899183    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:25.899272 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.899230    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 10:32:25.899272 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.899263    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found"
Jul 16 10:32:25.899850 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.899835    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 10:32:25.899850 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.899846    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 10:32:25.899985 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.899866    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 10:32:25.899985 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.899865    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 10:32:25.899985 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.899892    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID"
Jul 16 10:32:25.899985 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.899877    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID"
Jul 16 10:32:25.901206 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.901189    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:25.901293 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.901217    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 10:32:25.901571 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.901552    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f9ed974eab8cf27c3da7a3932d3ce15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal\" (UID: \"1f9ed974eab8cf27c3da7a3932d3ce15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:25.901652 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.901577    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f9ed974eab8cf27c3da7a3932d3ce15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal\" (UID: \"1f9ed974eab8cf27c3da7a3932d3ce15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:25.901652 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.901600    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/09d3c0959bdc2c44d35e2b9b5a065fd9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-213.ec2.internal\" (UID: \"09d3c0959bdc2c44d35e2b9b5a065fd9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:25.901926 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.901910    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 10:32:25.901990 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.901937    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 10:32:25.901990 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:25.901952    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeHasSufficientPID"
Jul 16 10:32:25.924547 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.924526    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-213.ec2.internal\" not found" node="ip-10-0-128-213.ec2.internal"
Jul 16 10:32:25.929388 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.929372    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-213.ec2.internal\" not found" node="ip-10-0-128-213.ec2.internal"
Jul 16 10:32:25.999881 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:25.999845    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found"
Jul 16 10:32:26.002235 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.002217    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f9ed974eab8cf27c3da7a3932d3ce15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal\" (UID: \"1f9ed974eab8cf27c3da7a3932d3ce15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:26.002321 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.002251    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f9ed974eab8cf27c3da7a3932d3ce15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal\" (UID: \"1f9ed974eab8cf27c3da7a3932d3ce15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:26.002321 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.002278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/09d3c0959bdc2c44d35e2b9b5a065fd9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-213.ec2.internal\" (UID: \"09d3c0959bdc2c44d35e2b9b5a065fd9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:26.002406 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.002337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f9ed974eab8cf27c3da7a3932d3ce15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal\" (UID: \"1f9ed974eab8cf27c3da7a3932d3ce15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:26.002406 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.002337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/09d3c0959bdc2c44d35e2b9b5a065fd9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-213.ec2.internal\" (UID: \"09d3c0959bdc2c44d35e2b9b5a065fd9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:26.002406 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.002338    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f9ed974eab8cf27c3da7a3932d3ce15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal\" (UID: \"1f9ed974eab8cf27c3da7a3932d3ce15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:26.100930 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:26.100860    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found"
Jul 16 10:32:26.201104 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:26.201075    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found"
Jul 16 10:32:26.227278 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.227251    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:26.232559 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.232509    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:26.302233 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:26.302186    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found"
Jul 16 10:32:26.406281 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:26.406216    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found"
Jul 16 10:32:26.507382 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:26.507331    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found"
Jul 16 10:32:26.539185 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.539147    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 16 10:32:26.539779 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.539310    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 10:32:26.607440 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:26.607410    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-213.ec2.internal\" not found"
Jul 16 10:32:26.617909 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.617875    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 10:27:26 +0000 UTC" deadline="2028-03-11 06:43:26.149540191 +0000 UTC"
Jul 16 10:32:26.617909 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.617901    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14492h10m59.53164234s"
Jul 16 10:32:26.658935 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.658870    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 10:32:26.700138 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.700106    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:26.715202 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.715174    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 10:32:26.718108 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.718093    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal"
Jul 16 10:32:26.730067 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.730043    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 10:32:26.807691 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.807669    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 10:32:26.808185 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:26.808159    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9ed974eab8cf27c3da7a3932d3ce15.slice/crio-955fda0a5caa98caa513993c2b142083a090201548e2203434b66f17dc47b3f8 WatchSource:0}: Error finding container 955fda0a5caa98caa513993c2b142083a090201548e2203434b66f17dc47b3f8: Status 404 returned error can't find the container with id 955fda0a5caa98caa513993c2b142083a090201548e2203434b66f17dc47b3f8
Jul 16 10:32:26.808874 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:26.808852    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09d3c0959bdc2c44d35e2b9b5a065fd9.slice/crio-efdaf43c692e17ea945bd6ab00c9ca60eb41f344d426af8c62da57352e4abefe WatchSource:0}: Error finding container efdaf43c692e17ea945bd6ab00c9ca60eb41f344d426af8c62da57352e4abefe: Status 404 returned error can't find the container with id efdaf43c692e17ea945bd6ab00c9ca60eb41f344d426af8c62da57352e4abefe
Jul 16 10:32:26.814195 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.814171    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 10:32:26.921006 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:26.920926    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 10:32:27.581874 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.581836    2576 apiserver.go:52] "Watching apiserver"
Jul 16 10:32:27.589573 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.589549    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 16 10:32:27.589996 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.589971    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-r8z79","kube-system/konnectivity-agent-lllwn","kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal","openshift-cluster-node-tuning-operator/tuned-vbmrn","openshift-image-registry/node-ca-d5zm6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal","openshift-multus/multus-additional-cni-plugins-drv8w","openshift-multus/multus-p7s2w","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9","openshift-dns/node-resolver-7l8m9","openshift-multus/network-metrics-daemon-kq258","openshift-network-diagnostics/network-check-target-pxkgn","openshift-network-operator/iptables-alerter-mt4m4","openshift-ovn-kubernetes/ovnkube-node-7pfmf"]
Jul 16 10:32:27.592642 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.592625    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.594982 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.594961    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mt4m4"
Jul 16 10:32:27.596895 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.596872    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 16 10:32:27.597091 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.597071    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 16 10:32:27.597176 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.597158    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 16 10:32:27.597370 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.597349    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hrn99\""
Jul 16 10:32:27.597482 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.597428    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.597560 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.597494    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 16 10:32:27.597878 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.597654    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 16 10:32:27.598074 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.598055    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 16 10:32:27.598183 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.598094    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 16 10:32:27.598377 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.598355    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 16 10:32:27.598482 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.598431    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-m9vt4\""
Jul 16 10:32:27.599849 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.599825    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.600238 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.600221    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 16 10:32:27.603812 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.602967    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 16 10:32:27.603812 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.603005    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 16 10:32:27.603812 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.603448    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 16 10:32:27.603812 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.603678    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 16 10:32:27.604063 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.603939    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 16 10:32:27.604063 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.604006    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 16 10:32:27.604063 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.604035    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-sn4g7\""
Jul 16 10:32:27.605470 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.604673    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 16 10:32:27.605470 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.605027    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-crxpx\""
Jul 16 10:32:27.605470 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.605161    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 16 10:32:27.605470 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.605348    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d5zm6"
Jul 16 10:32:27.607880 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.607859    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 16 10:32:27.608060 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.608007    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 16 10:32:27.608146 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.608129    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.608444 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.608348    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 16 10:32:27.608703 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.608683    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rkxvk\""
Jul 16 10:32:27.610723 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.610703    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kjhsp\""
Jul 16 10:32:27.610831 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.610754    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.611187 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.611070    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 16 10:32:27.613573 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.613549    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:27.613669 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.613601    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 16 10:32:27.613669 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:27.613632    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:27.613811 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.613700    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 16 10:32:27.613946 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.613929    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 16 10:32:27.614216 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614180    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-c69cb\""
Jul 16 10:32:27.614313 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5a80df04-e68b-404c-9ed7-96de4566f8a0-ovn-node-metrics-cert\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.614313 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614260    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-tuned\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.614313 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614293    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b89fcd0-9c09-475f-96b7-8f4029d612e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.614486 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-slash\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.614486 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614377    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-systemd\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.614486 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614405    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-sys\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.614486 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-lib-modules\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.614486 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614474    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhx47\" (UniqueName: \"kubernetes.io/projected/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-kube-api-access-fhx47\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.614720 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614500    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2aeb2209-a85b-460e-a65c-cd36e86a5a8e-serviceca\") pod \"node-ca-d5zm6\" (UID: \"2aeb2209-a85b-460e-a65c-cd36e86a5a8e\") " pod="openshift-image-registry/node-ca-d5zm6"
Jul 16 10:32:27.614720 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614500    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 10:32:27.614720 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b89fcd0-9c09-475f-96b7-8f4029d612e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.614720 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614569    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b89fcd0-9c09-475f-96b7-8f4029d612e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.614720 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg4nt\" (UniqueName: \"kubernetes.io/projected/1b89fcd0-9c09-475f-96b7-8f4029d612e2-kube-api-access-kg4nt\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.614720 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614622    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-var-lib-openvswitch\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.614720 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-log-socket\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.614720 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614671    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5a80df04-e68b-404c-9ed7-96de4566f8a0-ovnkube-config\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.614720 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614685    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-host\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614767    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614815    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2aeb2209-a85b-460e-a65c-cd36e86a5a8e-host\") pod \"node-ca-d5zm6\" (UID: \"2aeb2209-a85b-460e-a65c-cd36e86a5a8e\") " pod="openshift-image-registry/node-ca-d5zm6"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614839    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da75938e-6c0f-4dbb-b59e-61499fcd8a56-iptables-alerter-script\") pod \"iptables-alerter-mt4m4\" (UID: \"da75938e-6c0f-4dbb-b59e-61499fcd8a56\") " pod="openshift-network-operator/iptables-alerter-mt4m4"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614863    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-kubelet\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5a80df04-e68b-404c-9ed7-96de4566f8a0-ovnkube-script-lib\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614909    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-tmp\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614932    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hn8g\" (UniqueName: \"kubernetes.io/projected/2aeb2209-a85b-460e-a65c-cd36e86a5a8e-kube-api-access-8hn8g\") pod \"node-ca-d5zm6\" (UID: \"2aeb2209-a85b-460e-a65c-cd36e86a5a8e\") " pod="openshift-image-registry/node-ca-d5zm6"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.614956    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da75938e-6c0f-4dbb-b59e-61499fcd8a56-host-slash\") pod \"iptables-alerter-mt4m4\" (UID: \"da75938e-6c0f-4dbb-b59e-61499fcd8a56\") " pod="openshift-network-operator/iptables-alerter-mt4m4"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615005    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-run-ovn\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615055    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-sysconfig\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615082    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-kubernetes\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615110    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b89fcd0-9c09-475f-96b7-8f4029d612e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615146    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcc28\" (UniqueName: \"kubernetes.io/projected/da75938e-6c0f-4dbb-b59e-61499fcd8a56-kube-api-access-qcc28\") pod \"iptables-alerter-mt4m4\" (UID: \"da75938e-6c0f-4dbb-b59e-61499fcd8a56\") " pod="openshift-network-operator/iptables-alerter-mt4m4"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615171    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-run-netns\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.615212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615195    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-node-log\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615238    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-cni-netd\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615262    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-sysctl-d\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615286    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-sysctl-conf\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615310    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-var-lib-kubelet\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615388    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b89fcd0-9c09-475f-96b7-8f4029d612e2-system-cni-dir\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615434    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-run-ovn-kubernetes\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615456    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-cni-bin\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615475    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5a80df04-e68b-404c-9ed7-96de4566f8a0-env-overrides\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615499    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg9br\" (UniqueName: \"kubernetes.io/projected/5a80df04-e68b-404c-9ed7-96de4566f8a0-kube-api-access-gg9br\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615522    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-modprobe-d\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-run\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b89fcd0-9c09-475f-96b7-8f4029d612e2-cnibin\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b89fcd0-9c09-475f-96b7-8f4029d612e2-os-release\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615715    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-systemd-units\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615742    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-run-systemd\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.616007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615781    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-etc-openvswitch\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.616812 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.615814    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-run-openvswitch\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.616812 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.616188    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lllwn"
Jul 16 10:32:27.619316 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.619282    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7l8m9"
Jul 16 10:32:27.619758 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.619738    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 16 10:32:27.619906 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.619870    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jtz2h\""
Jul 16 10:32:27.620094 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.620078    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 16 10:32:27.621957 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.621934    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 16 10:32:27.622091 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.622073    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fqf46\""
Jul 16 10:32:27.622165 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.622143    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 16 10:32:27.622407 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.622388    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:27.622533 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:27.622511    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:27.625271 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.625252    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:27.625356 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:27.625322    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:27.648029 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.648007    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w588r"
Jul 16 10:32:27.658018 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.657986    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w588r"
Jul 16 10:32:27.701910 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.701885    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 16 10:32:27.716192 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhx47\" (UniqueName: \"kubernetes.io/projected/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-kube-api-access-fhx47\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.716378 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716201    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.716378 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716232    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b89fcd0-9c09-475f-96b7-8f4029d612e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.716378 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716261    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcc28\" (UniqueName: \"kubernetes.io/projected/da75938e-6c0f-4dbb-b59e-61499fcd8a56-kube-api-access-qcc28\") pod \"iptables-alerter-mt4m4\" (UID: \"da75938e-6c0f-4dbb-b59e-61499fcd8a56\") " pod="openshift-network-operator/iptables-alerter-mt4m4"
Jul 16 10:32:27.716378 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716294    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-sysctl-d\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.716378 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716322    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pclnp\" (UniqueName: \"kubernetes.io/projected/307f1116-bf81-4af7-92a8-f65cdf424f0d-kube-api-access-pclnp\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.716378 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b89fcd0-9c09-475f-96b7-8f4029d612e2-system-cni-dir\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.716378 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716374    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-run-ovn-kubernetes\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716399    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-cni-bin\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716421    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5a80df04-e68b-404c-9ed7-96de4566f8a0-env-overrides\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg9br\" (UniqueName: \"kubernetes.io/projected/5a80df04-e68b-404c-9ed7-96de4566f8a0-kube-api-access-gg9br\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716465    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-run\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716488    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b010b9df-7a02-4080-a99f-bd1160bc5f5b-cni-binary-copy\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716488    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-run-ovn-kubernetes\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716515    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-var-lib-cni-multus\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716524    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-sysctl-d\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716556    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b89fcd0-9c09-475f-96b7-8f4029d612e2-system-cni-dir\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716579    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-cni-bin\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716590    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-run-systemd\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716635    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-run\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-etc-openvswitch\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716675    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-run-systemd\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716681    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-run-openvswitch\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716713    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-etc-openvswitch\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.716756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5a80df04-e68b-404c-9ed7-96de4566f8a0-ovn-node-metrics-cert\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716752    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-run-openvswitch\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716759    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-systemd\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-lib-modules\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716846    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-run-k8s-cni-cncf-io\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716882    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-run-multus-certs\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716923    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg4nt\" (UniqueName: \"kubernetes.io/projected/1b89fcd0-9c09-475f-96b7-8f4029d612e2-kube-api-access-kg4nt\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716954    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b89fcd0-9c09-475f-96b7-8f4029d612e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716973    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-log-socket\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.716980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-lib-modules\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5a80df04-e68b-404c-9ed7-96de4566f8a0-ovnkube-config\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717100    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5a80df04-e68b-404c-9ed7-96de4566f8a0-env-overrides\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-systemd\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-log-socket\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717193    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-host\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717192    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717246    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-host\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717241    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-socket-dir\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.717697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717282    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f4f6727-e31d-4ec2-8dd0-91fcb059faf8-agent-certs\") pod \"konnectivity-agent-lllwn\" (UID: \"7f4f6727-e31d-4ec2-8dd0-91fcb059faf8\") " pod="kube-system/konnectivity-agent-lllwn"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717311    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-cnibin\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717336    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-hostroot\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717391    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2aeb2209-a85b-460e-a65c-cd36e86a5a8e-host\") pod \"node-ca-d5zm6\" (UID: \"2aeb2209-a85b-460e-a65c-cd36e86a5a8e\") " pod="openshift-image-registry/node-ca-d5zm6"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717430    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-etc-selinux\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717442    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5a80df04-e68b-404c-9ed7-96de4566f8a0-ovnkube-config\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2aeb2209-a85b-460e-a65c-cd36e86a5a8e-host\") pod \"node-ca-d5zm6\" (UID: \"2aeb2209-a85b-460e-a65c-cd36e86a5a8e\") " pod="openshift-image-registry/node-ca-d5zm6"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717464    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs\") pod \"network-metrics-daemon-kq258\" (UID: \"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f\") " pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717492    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b010b9df-7a02-4080-a99f-bd1160bc5f5b-multus-daemon-config\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-kubelet\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717549    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hn8g\" (UniqueName: \"kubernetes.io/projected/2aeb2209-a85b-460e-a65c-cd36e86a5a8e-kube-api-access-8hn8g\") pod \"node-ca-d5zm6\" (UID: \"2aeb2209-a85b-460e-a65c-cd36e86a5a8e\") " pod="openshift-image-registry/node-ca-d5zm6"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717596    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-kubelet\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717766    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3cc8c24-4def-40f0-9ee3-df7f26fdb61a-tmp-dir\") pod \"node-resolver-7l8m9\" (UID: \"d3cc8c24-4def-40f0-9ee3-df7f26fdb61a\") " pod="openshift-dns/node-resolver-7l8m9"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717795    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhcgs\" (UniqueName: \"kubernetes.io/projected/d3cc8c24-4def-40f0-9ee3-df7f26fdb61a-kube-api-access-qhcgs\") pod \"node-resolver-7l8m9\" (UID: \"d3cc8c24-4def-40f0-9ee3-df7f26fdb61a\") " pod="openshift-dns/node-resolver-7l8m9"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717832    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da75938e-6c0f-4dbb-b59e-61499fcd8a56-host-slash\") pod \"iptables-alerter-mt4m4\" (UID: \"da75938e-6c0f-4dbb-b59e-61499fcd8a56\") " pod="openshift-network-operator/iptables-alerter-mt4m4"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-var-lib-cni-bin\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ph5b\" (UniqueName: \"kubernetes.io/projected/b010b9df-7a02-4080-a99f-bd1160bc5f5b-kube-api-access-5ph5b\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.718564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717904    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da75938e-6c0f-4dbb-b59e-61499fcd8a56-host-slash\") pod \"iptables-alerter-mt4m4\" (UID: \"da75938e-6c0f-4dbb-b59e-61499fcd8a56\") " pod="openshift-network-operator/iptables-alerter-mt4m4"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717919    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2aeb2209-a85b-460e-a65c-cd36e86a5a8e-serviceca\") pod \"node-ca-d5zm6\" (UID: \"2aeb2209-a85b-460e-a65c-cd36e86a5a8e\") " pod="openshift-image-registry/node-ca-d5zm6"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717954    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-registration-dir\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.717982    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-device-dir\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-run-netns\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718019    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-node-log\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718044    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-cni-netd\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-run-netns\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718087    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-cni-netd\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-sysctl-conf\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718132    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-var-lib-kubelet\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718153    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-tuned\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718176    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-modprobe-d\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718088    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-node-log\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718200    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-multus-cni-dir\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718218    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-sysctl-conf\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-var-lib-kubelet\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.719432 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-multus-socket-dir-parent\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718275    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-var-lib-kubelet\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718305    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-modprobe-d\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718304    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718344    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2aeb2209-a85b-460e-a65c-cd36e86a5a8e-serviceca\") pod \"node-ca-d5zm6\" (UID: \"2aeb2209-a85b-460e-a65c-cd36e86a5a8e\") " pod="openshift-image-registry/node-ca-d5zm6"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718368    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b89fcd0-9c09-475f-96b7-8f4029d612e2-cnibin\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b89fcd0-9c09-475f-96b7-8f4029d612e2-cnibin\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718393    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b89fcd0-9c09-475f-96b7-8f4029d612e2-os-release\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718417    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-systemd-units\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718445    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f4f6727-e31d-4ec2-8dd0-91fcb059faf8-konnectivity-ca\") pod \"konnectivity-agent-lllwn\" (UID: \"7f4f6727-e31d-4ec2-8dd0-91fcb059faf8\") " pod="kube-system/konnectivity-agent-lllwn"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718458    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b89fcd0-9c09-475f-96b7-8f4029d612e2-os-release\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718473    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvmgl\" (UniqueName: \"kubernetes.io/projected/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-kube-api-access-bvmgl\") pod \"network-metrics-daemon-kq258\" (UID: \"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f\") " pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718484    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-systemd-units\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718502    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/30449683-8c30-47cf-a574-474beccc52b0-dbus\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718542    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b89fcd0-9c09-475f-96b7-8f4029d612e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718580    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-slash\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718603    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-sys\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.720281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718629    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-sys-fs\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718655    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-system-cni-dir\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718680    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b89fcd0-9c09-475f-96b7-8f4029d612e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718680    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-slash\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718710    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b89fcd0-9c09-475f-96b7-8f4029d612e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718731    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-sys\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718738    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-var-lib-openvswitch\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-multus-conf-dir\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718808    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/30449683-8c30-47cf-a574-474beccc52b0-kubelet-config\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718879    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da75938e-6c0f-4dbb-b59e-61499fcd8a56-iptables-alerter-script\") pod \"iptables-alerter-mt4m4\" (UID: \"da75938e-6c0f-4dbb-b59e-61499fcd8a56\") " pod="openshift-network-operator/iptables-alerter-mt4m4"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718882    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b89fcd0-9c09-475f-96b7-8f4029d612e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5a80df04-e68b-404c-9ed7-96de4566f8a0-ovnkube-script-lib\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-tmp\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718939    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3cc8c24-4def-40f0-9ee3-df7f26fdb61a-hosts-file\") pod \"node-resolver-7l8m9\" (UID: \"d3cc8c24-4def-40f0-9ee3-df7f26fdb61a\") " pod="openshift-dns/node-resolver-7l8m9"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.718984    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-os-release\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.721112 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-etc-kubernetes\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-run-ovn\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-sysconfig\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719090    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-kubernetes\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719091    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b89fcd0-9c09-475f-96b7-8f4029d612e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb979\" (UniqueName: \"kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979\") pod \"network-check-target-pxkgn\" (UID: \"78c8ab20-3864-469a-8af5-6e9778297bce\") " pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-run-netns\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719191    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-kubernetes\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719209    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-sysconfig\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719254    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-run-ovn\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719300    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5a80df04-e68b-404c-9ed7-96de4566f8a0-var-lib-openvswitch\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719452    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5a80df04-e68b-404c-9ed7-96de4566f8a0-ovnkube-script-lib\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719503    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da75938e-6c0f-4dbb-b59e-61499fcd8a56-iptables-alerter-script\") pod \"iptables-alerter-mt4m4\" (UID: \"da75938e-6c0f-4dbb-b59e-61499fcd8a56\") " pod="openshift-network-operator/iptables-alerter-mt4m4"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.719837    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b89fcd0-9c09-475f-96b7-8f4029d612e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.721272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5a80df04-e68b-404c-9ed7-96de4566f8a0-ovn-node-metrics-cert\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.721805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.721487    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-etc-tuned\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.722353 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.721858    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-tmp\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.727778 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.727743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcc28\" (UniqueName: \"kubernetes.io/projected/da75938e-6c0f-4dbb-b59e-61499fcd8a56-kube-api-access-qcc28\") pod \"iptables-alerter-mt4m4\" (UID: \"da75938e-6c0f-4dbb-b59e-61499fcd8a56\") " pod="openshift-network-operator/iptables-alerter-mt4m4"
Jul 16 10:32:27.727901 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.727872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhx47\" (UniqueName: \"kubernetes.io/projected/8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2-kube-api-access-fhx47\") pod \"tuned-vbmrn\" (UID: \"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2\") " pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.728046 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.728028    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg4nt\" (UniqueName: \"kubernetes.io/projected/1b89fcd0-9c09-475f-96b7-8f4029d612e2-kube-api-access-kg4nt\") pod \"multus-additional-cni-plugins-drv8w\" (UID: \"1b89fcd0-9c09-475f-96b7-8f4029d612e2\") " pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.728705 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.728682    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg9br\" (UniqueName: \"kubernetes.io/projected/5a80df04-e68b-404c-9ed7-96de4566f8a0-kube-api-access-gg9br\") pod \"ovnkube-node-7pfmf\" (UID: \"5a80df04-e68b-404c-9ed7-96de4566f8a0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.728806 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.728733    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hn8g\" (UniqueName: \"kubernetes.io/projected/2aeb2209-a85b-460e-a65c-cd36e86a5a8e-kube-api-access-8hn8g\") pod \"node-ca-d5zm6\" (UID: \"2aeb2209-a85b-460e-a65c-cd36e86a5a8e\") " pod="openshift-image-registry/node-ca-d5zm6"
Jul 16 10:32:27.801212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.801159    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" event={"ID":"1f9ed974eab8cf27c3da7a3932d3ce15","Type":"ContainerStarted","Data":"955fda0a5caa98caa513993c2b142083a090201548e2203434b66f17dc47b3f8"}
Jul 16 10:32:27.802469 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.802442    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal" event={"ID":"09d3c0959bdc2c44d35e2b9b5a065fd9","Type":"ContainerStarted","Data":"efdaf43c692e17ea945bd6ab00c9ca60eb41f344d426af8c62da57352e4abefe"}
Jul 16 10:32:27.819814 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.819790    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-var-lib-cni-bin\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.819935 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.819822    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ph5b\" (UniqueName: \"kubernetes.io/projected/b010b9df-7a02-4080-a99f-bd1160bc5f5b-kube-api-access-5ph5b\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.819935 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.819843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-registration-dir\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.819935 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.819869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-device-dir\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.819935 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.819898    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-var-lib-cni-bin\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.819935 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.819899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-multus-cni-dir\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.820129 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.819951    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-multus-socket-dir-parent\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.820129 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.819968    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-multus-cni-dir\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.820129 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.819966    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-registration-dir\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.820129 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.819978    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-var-lib-kubelet\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.820129 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820011    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-var-lib-kubelet\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.820129 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820022    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-device-dir\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.820129 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820031    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:27.820129 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820062    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-multus-socket-dir-parent\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.820129 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820067    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f4f6727-e31d-4ec2-8dd0-91fcb059faf8-konnectivity-ca\") pod \"konnectivity-agent-lllwn\" (UID: \"7f4f6727-e31d-4ec2-8dd0-91fcb059faf8\") " pod="kube-system/konnectivity-agent-lllwn"
Jul 16 10:32:27.820129 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820096    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvmgl\" (UniqueName: \"kubernetes.io/projected/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-kube-api-access-bvmgl\") pod \"network-metrics-daemon-kq258\" (UID: \"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f\") " pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:27.820129 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:27.820118    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:27.820129 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/30449683-8c30-47cf-a574-474beccc52b0-dbus\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820150    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-sys-fs\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820178    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-system-cni-dir\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:27.820208    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret podName:30449683-8c30-47cf-a574-474beccc52b0 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:28.320186587 +0000 UTC m=+3.057221868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret") pod "global-pull-secret-syncer-r8z79" (UID: "30449683-8c30-47cf-a574-474beccc52b0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820229    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-system-cni-dir\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-multus-conf-dir\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820269    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/30449683-8c30-47cf-a574-474beccc52b0-kubelet-config\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820305    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3cc8c24-4def-40f0-9ee3-df7f26fdb61a-hosts-file\") pod \"node-resolver-7l8m9\" (UID: \"d3cc8c24-4def-40f0-9ee3-df7f26fdb61a\") " pod="openshift-dns/node-resolver-7l8m9"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-os-release\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820354    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-etc-kubernetes\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820374    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/30449683-8c30-47cf-a574-474beccc52b0-dbus\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb979\" (UniqueName: \"kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979\") pod \"network-check-target-pxkgn\" (UID: \"78c8ab20-3864-469a-8af5-6e9778297bce\") " pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820409    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-sys-fs\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820412    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-run-netns\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820451    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-run-netns\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820455    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820486    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pclnp\" (UniqueName: \"kubernetes.io/projected/307f1116-bf81-4af7-92a8-f65cdf424f0d-kube-api-access-pclnp\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.820623 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b010b9df-7a02-4080-a99f-bd1160bc5f5b-cni-binary-copy\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820536    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-var-lib-cni-multus\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820563    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3cc8c24-4def-40f0-9ee3-df7f26fdb61a-hosts-file\") pod \"node-resolver-7l8m9\" (UID: \"d3cc8c24-4def-40f0-9ee3-df7f26fdb61a\") " pod="openshift-dns/node-resolver-7l8m9"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820569    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-run-k8s-cni-cncf-io\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820601    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-run-multus-certs\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820606    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-run-k8s-cni-cncf-io\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820425    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/30449683-8c30-47cf-a574-474beccc52b0-kubelet-config\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-socket-dir\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820661    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820665    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f4f6727-e31d-4ec2-8dd0-91fcb059faf8-agent-certs\") pod \"konnectivity-agent-lllwn\" (UID: \"7f4f6727-e31d-4ec2-8dd0-91fcb059faf8\") " pod="kube-system/konnectivity-agent-lllwn"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820694    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-cnibin\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820713    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f4f6727-e31d-4ec2-8dd0-91fcb059faf8-konnectivity-ca\") pod \"konnectivity-agent-lllwn\" (UID: \"7f4f6727-e31d-4ec2-8dd0-91fcb059faf8\") " pod="kube-system/konnectivity-agent-lllwn"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820721    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-hostroot\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820748    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-etc-selinux\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs\") pod \"network-metrics-daemon-kq258\" (UID: \"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f\") " pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820811    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-os-release\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820825    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b010b9df-7a02-4080-a99f-bd1160bc5f5b-multus-daemon-config\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820856    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3cc8c24-4def-40f0-9ee3-df7f26fdb61a-tmp-dir\") pod \"node-resolver-7l8m9\" (UID: \"d3cc8c24-4def-40f0-9ee3-df7f26fdb61a\") " pod="openshift-dns/node-resolver-7l8m9"
Jul 16 10:32:27.821399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820880    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-etc-kubernetes\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.822264 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820883    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhcgs\" (UniqueName: \"kubernetes.io/projected/d3cc8c24-4def-40f0-9ee3-df7f26fdb61a-kube-api-access-qhcgs\") pod \"node-resolver-7l8m9\" (UID: \"d3cc8c24-4def-40f0-9ee3-df7f26fdb61a\") " pod="openshift-dns/node-resolver-7l8m9"
Jul 16 10:32:27.822264 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820915    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-socket-dir\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.822264 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-run-multus-certs\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.822264 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820974    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-hostroot\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.822264 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:27.821037    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:27.822264 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.821096    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-host-var-lib-cni-multus\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.822264 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.821149    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-cnibin\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.822264 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:27.821158    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs podName:0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:28.321140823 +0000 UTC m=+3.058176076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs") pod "network-metrics-daemon-kq258" (UID: "0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:27.822264 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.821155    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/307f1116-bf81-4af7-92a8-f65cdf424f0d-etc-selinux\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.822264 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.820634    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b010b9df-7a02-4080-a99f-bd1160bc5f5b-multus-conf-dir\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.822264 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.821437    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3cc8c24-4def-40f0-9ee3-df7f26fdb61a-tmp-dir\") pod \"node-resolver-7l8m9\" (UID: \"d3cc8c24-4def-40f0-9ee3-df7f26fdb61a\") " pod="openshift-dns/node-resolver-7l8m9"
Jul 16 10:32:27.822264 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.821632    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b010b9df-7a02-4080-a99f-bd1160bc5f5b-cni-binary-copy\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.822264 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.821807    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b010b9df-7a02-4080-a99f-bd1160bc5f5b-multus-daemon-config\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.823564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.823543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f4f6727-e31d-4ec2-8dd0-91fcb059faf8-agent-certs\") pod \"konnectivity-agent-lllwn\" (UID: \"7f4f6727-e31d-4ec2-8dd0-91fcb059faf8\") " pod="kube-system/konnectivity-agent-lllwn"
Jul 16 10:32:27.832822 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:27.832744    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 10:32:27.832822 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:27.832786    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 10:32:27.832822 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:27.832800    2576 projected.go:196] Error preparing data for projected volume kube-api-access-rb979 for pod openshift-network-diagnostics/network-check-target-pxkgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:27.833015 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:27.832855    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979 podName:78c8ab20-3864-469a-8af5-6e9778297bce nodeName:}" failed. No retries permitted until 2026-07-16 10:32:28.33284124 +0000 UTC m=+3.069876492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rb979" (UniqueName: "kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979") pod "network-check-target-pxkgn" (UID: "78c8ab20-3864-469a-8af5-6e9778297bce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:27.835077 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.835057    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pclnp\" (UniqueName: \"kubernetes.io/projected/307f1116-bf81-4af7-92a8-f65cdf424f0d-kube-api-access-pclnp\") pod \"aws-ebs-csi-driver-node-2p9q9\" (UID: \"307f1116-bf81-4af7-92a8-f65cdf424f0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.835637 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.835617    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhcgs\" (UniqueName: \"kubernetes.io/projected/d3cc8c24-4def-40f0-9ee3-df7f26fdb61a-kube-api-access-qhcgs\") pod \"node-resolver-7l8m9\" (UID: \"d3cc8c24-4def-40f0-9ee3-df7f26fdb61a\") " pod="openshift-dns/node-resolver-7l8m9"
Jul 16 10:32:27.836109 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.836089    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ph5b\" (UniqueName: \"kubernetes.io/projected/b010b9df-7a02-4080-a99f-bd1160bc5f5b-kube-api-access-5ph5b\") pod \"multus-p7s2w\" (UID: \"b010b9df-7a02-4080-a99f-bd1160bc5f5b\") " pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.836165 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.836128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvmgl\" (UniqueName: \"kubernetes.io/projected/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-kube-api-access-bvmgl\") pod \"network-metrics-daemon-kq258\" (UID: \"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f\") " pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:27.909723 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.909688    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-drv8w"
Jul 16 10:32:27.918539 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.918514    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mt4m4"
Jul 16 10:32:27.926663 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.926637    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:27.934004 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.933978    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vbmrn"
Jul 16 10:32:27.940888 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.940870    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d5zm6"
Jul 16 10:32:27.948724 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.948706    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p7s2w"
Jul 16 10:32:27.956679 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.956652    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9"
Jul 16 10:32:27.968636 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.968615    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lllwn"
Jul 16 10:32:27.973537 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:27.973518    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7l8m9"
Jul 16 10:32:28.040378 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.040353    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 10:32:28.323962 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.323874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:28.324140 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.323968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs\") pod \"network-metrics-daemon-kq258\" (UID: \"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f\") " pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:28.324140 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:28.324052    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:28.324140 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:28.324111    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:28.324140 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:28.324134    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret podName:30449683-8c30-47cf-a574-474beccc52b0 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:29.324110791 +0000 UTC m=+4.061146064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret") pod "global-pull-secret-syncer-r8z79" (UID: "30449683-8c30-47cf-a574-474beccc52b0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:28.324329 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:28.324174    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs podName:0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:29.324155915 +0000 UTC m=+4.061191182 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs") pod "network-metrics-daemon-kq258" (UID: "0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:28.387239 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:28.387204    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb010b9df_7a02_4080_a99f_bd1160bc5f5b.slice/crio-69e652b567b63ae602d51587de813c0d48a5708f314ca2706aadcef82ccfcf83 WatchSource:0}: Error finding container 69e652b567b63ae602d51587de813c0d48a5708f314ca2706aadcef82ccfcf83: Status 404 returned error can't find the container with id 69e652b567b63ae602d51587de813c0d48a5708f314ca2706aadcef82ccfcf83
Jul 16 10:32:28.392993 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:28.392971    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b89fcd0_9c09_475f_96b7_8f4029d612e2.slice/crio-268387152a6c1637abd3ba534702ce8a0684e25d21c81f9d9ab3ea46a03939ce WatchSource:0}: Error finding container 268387152a6c1637abd3ba534702ce8a0684e25d21c81f9d9ab3ea46a03939ce: Status 404 returned error can't find the container with id 268387152a6c1637abd3ba534702ce8a0684e25d21c81f9d9ab3ea46a03939ce
Jul 16 10:32:28.393865 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:28.393838    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda75938e_6c0f_4dbb_b59e_61499fcd8a56.slice/crio-eed735df2280fcba81609a37daab9c2d7529015a73beb32d195996971fc9680b WatchSource:0}: Error finding container eed735df2280fcba81609a37daab9c2d7529015a73beb32d195996971fc9680b: Status 404 returned error can't find the container with id eed735df2280fcba81609a37daab9c2d7529015a73beb32d195996971fc9680b
Jul 16 10:32:28.394786 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:28.394728    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f4f6727_e31d_4ec2_8dd0_91fcb059faf8.slice/crio-1a831b4e677314459dc06545b91bfcaac344c2910c4676790dd3a9b2b58f9b95 WatchSource:0}: Error finding container 1a831b4e677314459dc06545b91bfcaac344c2910c4676790dd3a9b2b58f9b95: Status 404 returned error can't find the container with id 1a831b4e677314459dc06545b91bfcaac344c2910c4676790dd3a9b2b58f9b95
Jul 16 10:32:28.395654 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:28.395626    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod307f1116_bf81_4af7_92a8_f65cdf424f0d.slice/crio-3899513af1438f6a7877ba2efc669b0ef60dc3dcf17cf5a69ceef3e87563270b WatchSource:0}: Error finding container 3899513af1438f6a7877ba2efc669b0ef60dc3dcf17cf5a69ceef3e87563270b: Status 404 returned error can't find the container with id 3899513af1438f6a7877ba2efc669b0ef60dc3dcf17cf5a69ceef3e87563270b
Jul 16 10:32:28.397462 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:28.397299    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8324ed0e_1ab0_4ae0_a5c7_c2fa1744b5d2.slice/crio-e81518e499c1bc32f1a89c530bbf37e46e79fd32d7e12dae4971801252fac4b6 WatchSource:0}: Error finding container e81518e499c1bc32f1a89c530bbf37e46e79fd32d7e12dae4971801252fac4b6: Status 404 returned error can't find the container with id e81518e499c1bc32f1a89c530bbf37e46e79fd32d7e12dae4971801252fac4b6
Jul 16 10:32:28.398734 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:28.398690    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aeb2209_a85b_460e_a65c_cd36e86a5a8e.slice/crio-a16cd21ea26a6b5efd749e688e4da706e28accf2437439eb63cd80b12bbe0ee0 WatchSource:0}: Error finding container a16cd21ea26a6b5efd749e688e4da706e28accf2437439eb63cd80b12bbe0ee0: Status 404 returned error can't find the container with id a16cd21ea26a6b5efd749e688e4da706e28accf2437439eb63cd80b12bbe0ee0
Jul 16 10:32:28.399839 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:28.399489    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a80df04_e68b_404c_9ed7_96de4566f8a0.slice/crio-c9c9e9e9015ccb3254fa34febddbf74641c9643aeb04b1ee43659bd301f96026 WatchSource:0}: Error finding container c9c9e9e9015ccb3254fa34febddbf74641c9643aeb04b1ee43659bd301f96026: Status 404 returned error can't find the container with id c9c9e9e9015ccb3254fa34febddbf74641c9643aeb04b1ee43659bd301f96026
Jul 16 10:32:28.401133 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:32:28.401101    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3cc8c24_4def_40f0_9ee3_df7f26fdb61a.slice/crio-9c35e384f03926a1e4e301bd47609c7ce12a9111bef012d5eac7a53f37b747a9 WatchSource:0}: Error finding container 9c35e384f03926a1e4e301bd47609c7ce12a9111bef012d5eac7a53f37b747a9: Status 404 returned error can't find the container with id 9c35e384f03926a1e4e301bd47609c7ce12a9111bef012d5eac7a53f37b747a9
Jul 16 10:32:28.424286 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.424251    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb979\" (UniqueName: \"kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979\") pod \"network-check-target-pxkgn\" (UID: \"78c8ab20-3864-469a-8af5-6e9778297bce\") " pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:28.424406 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:28.424383    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 10:32:28.424406 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:28.424399    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 10:32:28.424406 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:28.424409    2576 projected.go:196] Error preparing data for projected volume kube-api-access-rb979 for pod openshift-network-diagnostics/network-check-target-pxkgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:28.424933 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:28.424463    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979 podName:78c8ab20-3864-469a-8af5-6e9778297bce nodeName:}" failed. No retries permitted until 2026-07-16 10:32:29.424445268 +0000 UTC m=+4.161480519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rb979" (UniqueName: "kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979") pod "network-check-target-pxkgn" (UID: "78c8ab20-3864-469a-8af5-6e9778297bce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:28.662283 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.662001    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 10:27:27 +0000 UTC" deadline="2028-02-11 01:07:56.687779273 +0000 UTC"
Jul 16 10:32:28.662283 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.662206    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13790h35m28.025577184s"
Jul 16 10:32:28.806021 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.805909    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d5zm6" event={"ID":"2aeb2209-a85b-460e-a65c-cd36e86a5a8e","Type":"ContainerStarted","Data":"a16cd21ea26a6b5efd749e688e4da706e28accf2437439eb63cd80b12bbe0ee0"}
Jul 16 10:32:28.807417 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.807389    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vbmrn" event={"ID":"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2","Type":"ContainerStarted","Data":"e81518e499c1bc32f1a89c530bbf37e46e79fd32d7e12dae4971801252fac4b6"}
Jul 16 10:32:28.808974 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.808937    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lllwn" event={"ID":"7f4f6727-e31d-4ec2-8dd0-91fcb059faf8","Type":"ContainerStarted","Data":"1a831b4e677314459dc06545b91bfcaac344c2910c4676790dd3a9b2b58f9b95"}
Jul 16 10:32:28.810644 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.810616    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-drv8w" event={"ID":"1b89fcd0-9c09-475f-96b7-8f4029d612e2","Type":"ContainerStarted","Data":"268387152a6c1637abd3ba534702ce8a0684e25d21c81f9d9ab3ea46a03939ce"}
Jul 16 10:32:28.812163 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.812123    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p7s2w" event={"ID":"b010b9df-7a02-4080-a99f-bd1160bc5f5b","Type":"ContainerStarted","Data":"69e652b567b63ae602d51587de813c0d48a5708f314ca2706aadcef82ccfcf83"}
Jul 16 10:32:28.814971 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.814918    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal" event={"ID":"09d3c0959bdc2c44d35e2b9b5a065fd9","Type":"ContainerStarted","Data":"81a70d8f6a2ee8e45397fd097597c14b745dd3c0579b78add265942ba28881aa"}
Jul 16 10:32:28.816507 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.816450    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7l8m9" event={"ID":"d3cc8c24-4def-40f0-9ee3-df7f26fdb61a","Type":"ContainerStarted","Data":"9c35e384f03926a1e4e301bd47609c7ce12a9111bef012d5eac7a53f37b747a9"}
Jul 16 10:32:28.818039 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.818010    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf" event={"ID":"5a80df04-e68b-404c-9ed7-96de4566f8a0","Type":"ContainerStarted","Data":"c9c9e9e9015ccb3254fa34febddbf74641c9643aeb04b1ee43659bd301f96026"}
Jul 16 10:32:28.819938 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.819907    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9" event={"ID":"307f1116-bf81-4af7-92a8-f65cdf424f0d","Type":"ContainerStarted","Data":"3899513af1438f6a7877ba2efc669b0ef60dc3dcf17cf5a69ceef3e87563270b"}
Jul 16 10:32:28.821920 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.821894    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mt4m4" event={"ID":"da75938e-6c0f-4dbb-b59e-61499fcd8a56","Type":"ContainerStarted","Data":"eed735df2280fcba81609a37daab9c2d7529015a73beb32d195996971fc9680b"}
Jul 16 10:32:28.833685 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:28.833629    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-213.ec2.internal" podStartSLOduration=1.833610256 podStartE2EDuration="1.833610256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:28.833254582 +0000 UTC m=+3.570289859" watchObservedRunningTime="2026-07-16 10:32:28.833610256 +0000 UTC m=+3.570645531"
Jul 16 10:32:29.333273 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:29.333179    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs\") pod \"network-metrics-daemon-kq258\" (UID: \"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f\") " pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:29.333273 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:29.333250    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:29.333507 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:29.333429    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:29.333507 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:29.333497    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret podName:30449683-8c30-47cf-a574-474beccc52b0 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:31.333477865 +0000 UTC m=+6.070513117 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret") pod "global-pull-secret-syncer-r8z79" (UID: "30449683-8c30-47cf-a574-474beccc52b0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:29.333999 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:29.333978    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:29.334086 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:29.334035    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs podName:0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:31.334019095 +0000 UTC m=+6.071054352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs") pod "network-metrics-daemon-kq258" (UID: "0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:29.433836 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:29.433764    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb979\" (UniqueName: \"kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979\") pod \"network-check-target-pxkgn\" (UID: \"78c8ab20-3864-469a-8af5-6e9778297bce\") " pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:29.434043 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:29.434025    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 10:32:29.434106 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:29.434051    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 10:32:29.434106 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:29.434064    2576 projected.go:196] Error preparing data for projected volume kube-api-access-rb979 for pod openshift-network-diagnostics/network-check-target-pxkgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:29.434213 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:29.434130    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979 podName:78c8ab20-3864-469a-8af5-6e9778297bce nodeName:}" failed. No retries permitted until 2026-07-16 10:32:31.434104993 +0000 UTC m=+6.171140250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rb979" (UniqueName: "kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979") pod "network-check-target-pxkgn" (UID: "78c8ab20-3864-469a-8af5-6e9778297bce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:29.662492 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:29.662396    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 10:27:27 +0000 UTC" deadline="2028-01-24 15:29:32.026923154 +0000 UTC"
Jul 16 10:32:29.662492 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:29.662444    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13372h57m2.364487583s"
Jul 16 10:32:29.798369 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:29.798324    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:29.798555 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:29.798468    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:29.798858 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:29.798825    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:29.798966 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:29.798940    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:29.799031 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:29.799017    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:29.799134 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:29.799116    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:31.351893 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:31.351550    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs\") pod \"network-metrics-daemon-kq258\" (UID: \"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f\") " pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:31.351893 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:31.351607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:31.351893 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:31.351737    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:31.351893 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:31.351802    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret podName:30449683-8c30-47cf-a574-474beccc52b0 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:35.351782683 +0000 UTC m=+10.088817954 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret") pod "global-pull-secret-syncer-r8z79" (UID: "30449683-8c30-47cf-a574-474beccc52b0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:31.353578 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:31.353447    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:31.353578 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:31.353509    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs podName:0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:35.353490241 +0000 UTC m=+10.090525493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs") pod "network-metrics-daemon-kq258" (UID: "0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:31.452173 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:31.452130    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb979\" (UniqueName: \"kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979\") pod \"network-check-target-pxkgn\" (UID: \"78c8ab20-3864-469a-8af5-6e9778297bce\") " pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:31.452360 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:31.452316    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 10:32:31.452360 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:31.452342    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 10:32:31.452360 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:31.452354    2576 projected.go:196] Error preparing data for projected volume kube-api-access-rb979 for pod openshift-network-diagnostics/network-check-target-pxkgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:31.452528 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:31.452415    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979 podName:78c8ab20-3864-469a-8af5-6e9778297bce nodeName:}" failed. No retries permitted until 2026-07-16 10:32:35.452394796 +0000 UTC m=+10.189430073 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rb979" (UniqueName: "kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979") pod "network-check-target-pxkgn" (UID: "78c8ab20-3864-469a-8af5-6e9778297bce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:31.796016 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:31.795923    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:31.796016 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:31.795923    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:31.796218 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:31.796078    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:31.796274 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:31.796222    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:31.796322 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:31.796306    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:31.796448 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:31.796395    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:31.837302 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:31.837262    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" event={"ID":"1f9ed974eab8cf27c3da7a3932d3ce15","Type":"ContainerStarted","Data":"967da600eeaa0a390194f554fdd9e5ac1ae056b3906742b8196522023b1975b2"}
Jul 16 10:32:33.795890 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:33.795811    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:33.796356 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:33.795950    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:33.796356 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:33.795982    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:33.796356 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:33.796111    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:33.796356 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:33.796175    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:33.796356 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:33.796259    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:34.849919 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:34.849882    2576 generic.go:356] "Generic (PLEG): container finished" podID="1f9ed974eab8cf27c3da7a3932d3ce15" containerID="967da600eeaa0a390194f554fdd9e5ac1ae056b3906742b8196522023b1975b2" exitCode=0
Jul 16 10:32:34.850347 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:34.849937    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" event={"ID":"1f9ed974eab8cf27c3da7a3932d3ce15","Type":"ContainerDied","Data":"967da600eeaa0a390194f554fdd9e5ac1ae056b3906742b8196522023b1975b2"}
Jul 16 10:32:35.384925 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:35.384885    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:35.385115 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:35.384977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs\") pod \"network-metrics-daemon-kq258\" (UID: \"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f\") " pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:35.385115 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:35.385022    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:35.385115 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:35.385070    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:35.385115 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:35.385078    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret podName:30449683-8c30-47cf-a574-474beccc52b0 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:43.385059753 +0000 UTC m=+18.122095009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret") pod "global-pull-secret-syncer-r8z79" (UID: "30449683-8c30-47cf-a574-474beccc52b0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:35.385115 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:35.385103    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs podName:0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:43.385092304 +0000 UTC m=+18.122127569 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs") pod "network-metrics-daemon-kq258" (UID: "0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:35.486217 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:35.486175    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb979\" (UniqueName: \"kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979\") pod \"network-check-target-pxkgn\" (UID: \"78c8ab20-3864-469a-8af5-6e9778297bce\") " pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:35.486402 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:35.486332    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 10:32:35.486402 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:35.486353    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 10:32:35.486402 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:35.486366    2576 projected.go:196] Error preparing data for projected volume kube-api-access-rb979 for pod openshift-network-diagnostics/network-check-target-pxkgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:35.486556 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:35.486421    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979 podName:78c8ab20-3864-469a-8af5-6e9778297bce nodeName:}" failed. No retries permitted until 2026-07-16 10:32:43.486406875 +0000 UTC m=+18.223442127 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rb979" (UniqueName: "kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979") pod "network-check-target-pxkgn" (UID: "78c8ab20-3864-469a-8af5-6e9778297bce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:35.796857 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:35.796094    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:35.796857 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:35.796193    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:35.796857 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:35.796545    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:35.796857 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:35.796630    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:35.796857 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:35.796674    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:35.796857 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:35.796779    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:37.795897 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:37.795863    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:37.796364 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:37.796003    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:37.796425 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:37.796401    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:37.796506 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:37.796480    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:37.796568 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:37.796554    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:37.796656 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:37.796630    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:39.795804 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:39.795705    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:39.795804 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:39.795739    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:39.796302 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:39.795714    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:39.796302 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:39.795879    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:39.796302 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:39.795965    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:39.796302 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:39.796048    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:41.795811 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:41.795779    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:41.796257 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:41.795895    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:41.796257 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:41.795779    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:41.796257 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:41.795779    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:41.796257 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:41.796038    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:41.796257 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:41.796099    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:43.446632 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:43.446592    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs\") pod \"network-metrics-daemon-kq258\" (UID: \"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f\") " pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:43.447163 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:43.446653    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:43.447163 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:43.446770    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:43.447163 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:43.446774    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:43.447163 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:43.446829    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret podName:30449683-8c30-47cf-a574-474beccc52b0 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:59.446812401 +0000 UTC m=+34.183847673 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret") pod "global-pull-secret-syncer-r8z79" (UID: "30449683-8c30-47cf-a574-474beccc52b0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:43.447163 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:43.446847    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs podName:0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f nodeName:}" failed. No retries permitted until 2026-07-16 10:32:59.446839492 +0000 UTC m=+34.183874741 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs") pod "network-metrics-daemon-kq258" (UID: "0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:43.547800 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:43.547761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb979\" (UniqueName: \"kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979\") pod \"network-check-target-pxkgn\" (UID: \"78c8ab20-3864-469a-8af5-6e9778297bce\") " pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:43.547959 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:43.547921    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 10:32:43.547959 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:43.547941    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 10:32:43.547959 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:43.547950    2576 projected.go:196] Error preparing data for projected volume kube-api-access-rb979 for pod openshift-network-diagnostics/network-check-target-pxkgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:43.548074 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:43.547998    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979 podName:78c8ab20-3864-469a-8af5-6e9778297bce nodeName:}" failed. No retries permitted until 2026-07-16 10:32:59.547986088 +0000 UTC m=+34.285021337 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rb979" (UniqueName: "kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979") pod "network-check-target-pxkgn" (UID: "78c8ab20-3864-469a-8af5-6e9778297bce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:43.795379 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:43.795271    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:43.795379 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:43.795271    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:43.795596 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:43.795410    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:43.795596 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:43.795271    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:43.795596 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:43.795470    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:43.795596 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:43.795531    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:45.796178 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:45.796137    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:45.796656 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:45.796246    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:45.796656 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:45.796255    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:45.796656 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:45.796357    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:45.796656 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:45.796375    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:45.796656 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:45.796409    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:47.795082 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.794899    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:47.795675 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.794915    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:47.795675 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:47.795165    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:47.795675 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.794967    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:47.795675 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:47.795258    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:47.795675 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:47.795299    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:47.874065 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.874037    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" event={"ID":"1f9ed974eab8cf27c3da7a3932d3ce15","Type":"ContainerStarted","Data":"d7e023382bf19f570a0628269a7c9e574184a04a1dfd1a748cdf70f96a7f8199"}
Jul 16 10:32:47.875199 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.875178    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d5zm6" event={"ID":"2aeb2209-a85b-460e-a65c-cd36e86a5a8e","Type":"ContainerStarted","Data":"944055043b2dcfc92edd2ba04daf86f05b690560d0b332b3eee5eaab527acfc5"}
Jul 16 10:32:47.876680 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.876655    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vbmrn" event={"ID":"8324ed0e-1ab0-4ae0-a5c7-c2fa1744b5d2","Type":"ContainerStarted","Data":"554034010b15371fea576381100b8ec57919e96604ec396420c42adb2c6451b1"}
Jul 16 10:32:47.878053 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.878032    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lllwn" event={"ID":"7f4f6727-e31d-4ec2-8dd0-91fcb059faf8","Type":"ContainerStarted","Data":"cf838681ad78b46c0c013a017af0c948a75ba5f33dee52a52bbc47ed2f055970"}
Jul 16 10:32:47.879365 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.879341    2576 generic.go:356] "Generic (PLEG): container finished" podID="1b89fcd0-9c09-475f-96b7-8f4029d612e2" containerID="ddec712af66ae2de5cdaa3b81f785f22bc6b0205e11254a78839bd9444ce09e5" exitCode=0
Jul 16 10:32:47.879467 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.879367    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-drv8w" event={"ID":"1b89fcd0-9c09-475f-96b7-8f4029d612e2","Type":"ContainerDied","Data":"ddec712af66ae2de5cdaa3b81f785f22bc6b0205e11254a78839bd9444ce09e5"}
Jul 16 10:32:47.880838 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.880807    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p7s2w" event={"ID":"b010b9df-7a02-4080-a99f-bd1160bc5f5b","Type":"ContainerStarted","Data":"f9df8988a9b388bd4d7e7da9b64b5793c7d5a7be1302527f3a858737482415c0"}
Jul 16 10:32:47.882265 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.882248    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7l8m9" event={"ID":"d3cc8c24-4def-40f0-9ee3-df7f26fdb61a","Type":"ContainerStarted","Data":"67035e5eb1a206b601a71591bb886080286f662f7830437b41bd4aaaa6b6bcd6"}
Jul 16 10:32:47.885903 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.885888    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:32:47.886312 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.886295    2576 generic.go:356] "Generic (PLEG): container finished" podID="5a80df04-e68b-404c-9ed7-96de4566f8a0" containerID="3a64a1bac682258b83b1e93d9930c697cfaf2cb466310e949e7059b879571abb" exitCode=1
Jul 16 10:32:47.886438 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.886355    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf" event={"ID":"5a80df04-e68b-404c-9ed7-96de4566f8a0","Type":"ContainerStarted","Data":"f5623c32eb8ee3461754fdcb42cdc16b9b44c070594a8e850088738816aa5ebf"}
Jul 16 10:32:47.886438 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.886403    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf" event={"ID":"5a80df04-e68b-404c-9ed7-96de4566f8a0","Type":"ContainerDied","Data":"3a64a1bac682258b83b1e93d9930c697cfaf2cb466310e949e7059b879571abb"}
Jul 16 10:32:47.886527 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.886442    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf" event={"ID":"5a80df04-e68b-404c-9ed7-96de4566f8a0","Type":"ContainerStarted","Data":"4cb297d803c3c2b547690d5debbf38bc147c8e881ef372b81ddd6bf350431411"}
Jul 16 10:32:47.887985 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.887967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9" event={"ID":"307f1116-bf81-4af7-92a8-f65cdf424f0d","Type":"ContainerStarted","Data":"8c3e1fa55585bfca19e09dbb3efa6f94ca69e3579f6fd4201d2583762088bc2b"}
Jul 16 10:32:47.913329 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.913286    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-d5zm6" podStartSLOduration=2.880798768 podStartE2EDuration="21.913272307s" totalImagesPullingTime="19.032473539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:47.893390424 +0000 UTC m=+22.630425691" watchObservedRunningTime="2026-07-16 10:32:47.913272307 +0000 UTC m=+22.650307579"
Jul 16 10:32:47.913431 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.913353    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-213.ec2.internal" podStartSLOduration=20.913349779 podStartE2EDuration="20.913349779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:47.913228682 +0000 UTC m=+22.650263952" watchObservedRunningTime="2026-07-16 10:32:47.913349779 +0000 UTC m=+22.650385049"
Jul 16 10:32:47.944342 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.944300    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vbmrn" podStartSLOduration=2.911239554 podStartE2EDuration="21.944284998s" totalImagesPullingTime="19.033045444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:47.944131111 +0000 UTC m=+22.681166381" watchObservedRunningTime="2026-07-16 10:32:47.944284998 +0000 UTC m=+22.681320269"
Jul 16 10:32:47.984811 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.984639    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lllwn" podStartSLOduration=10.959539788 podStartE2EDuration="21.984623977s" totalImagesPullingTime="11.025084189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:47.98445517 +0000 UTC m=+22.721490439" watchObservedRunningTime="2026-07-16 10:32:47.984623977 +0000 UTC m=+22.721659261"
Jul 16 10:32:47.985114 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:47.985083    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-p7s2w" podStartSLOduration=2.861758292 podStartE2EDuration="21.985071865s" totalImagesPullingTime="19.123313573s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:47.96472736 +0000 UTC m=+22.701762631" watchObservedRunningTime="2026-07-16 10:32:47.985071865 +0000 UTC m=+22.722107135"
Jul 16 10:32:48.000130 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:48.000079    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7l8m9" podStartSLOduration=2.967549775 podStartE2EDuration="22.000060675s" totalImagesPullingTime="19.0325109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:47.999770638 +0000 UTC m=+22.736805908" watchObservedRunningTime="2026-07-16 10:32:48.000060675 +0000 UTC m=+22.737095947"
Jul 16 10:32:48.753055 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:48.753012    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 16 10:32:48.893841 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:48.893811    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:32:48.894261 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:48.894229    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf" event={"ID":"5a80df04-e68b-404c-9ed7-96de4566f8a0","Type":"ContainerStarted","Data":"71ca48dc171de4b7b2caa021cfad2166c1f323e284c43b6dbd69ce2ba2a5873f"}
Jul 16 10:32:48.894374 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:48.894357    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf" event={"ID":"5a80df04-e68b-404c-9ed7-96de4566f8a0","Type":"ContainerStarted","Data":"cf4f808bf1c8a4de2d8398aaebfb44c944d2f50619b05bcdff2e98ea51ba2644"}
Jul 16 10:32:48.894441 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:48.894413    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf" event={"ID":"5a80df04-e68b-404c-9ed7-96de4566f8a0","Type":"ContainerStarted","Data":"9645e4d90c9a21b37ddc79dcef94b617d8d7200f8ebc441611adaea6f209ee61"}
Jul 16 10:32:48.896269 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:48.896239    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9" event={"ID":"307f1116-bf81-4af7-92a8-f65cdf424f0d","Type":"ContainerStarted","Data":"38afe2da625d47be4b96a417150b4d503181071078f34c9290218ea29023cef7"}
Jul 16 10:32:49.677108 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:49.677002    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T10:32:48.753041813Z","UUID":"d8b1a866-8528-4d5c-a580-56ab6c44f810","Handler":null,"Name":"","Endpoint":""}
Jul 16 10:32:49.678871 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:49.678850    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 16 10:32:49.678995 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:49.678894    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 16 10:32:49.795237 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:49.795200    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:49.795237 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:49.795225    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:49.795476 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:49.795200    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:49.795476 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:49.795345    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:49.795476 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:49.795467    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:49.795641 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:49.795559    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:49.900323 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:49.900287    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mt4m4" event={"ID":"da75938e-6c0f-4dbb-b59e-61499fcd8a56","Type":"ContainerStarted","Data":"2af8389e1e57bb0edc8a497b86506e76522ee4bc5890f87addfb0b69fc7ec814"}
Jul 16 10:32:49.918834 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:49.918587    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mt4m4" podStartSLOduration=4.857814428 podStartE2EDuration="23.918568562s" totalImagesPullingTime="19.060754134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:49.917961933 +0000 UTC m=+24.654997204" watchObservedRunningTime="2026-07-16 10:32:49.918568562 +0000 UTC m=+24.655603836"
Jul 16 10:32:50.905812 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:50.905783    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:32:50.906268 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:50.906178    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf" event={"ID":"5a80df04-e68b-404c-9ed7-96de4566f8a0","Type":"ContainerStarted","Data":"c089a3a80ab11ca9093b7b29f5798a8c748fd30d6350bbd3438a88cb8fce0618"}
Jul 16 10:32:50.908317 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:50.908291    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9" event={"ID":"307f1116-bf81-4af7-92a8-f65cdf424f0d","Type":"ContainerStarted","Data":"72ee2ad02de85d68bf51d90b3e81a0b23ef3646dd332d65328c7e4c77bd55e33"}
Jul 16 10:32:50.932043 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:50.931997    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2p9q9" podStartSLOduration=3.891351335 podStartE2EDuration="24.931982021s" totalImagesPullingTime="21.040630686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:50.93146814 +0000 UTC m=+25.668503413" watchObservedRunningTime="2026-07-16 10:32:50.931982021 +0000 UTC m=+25.669017293"
Jul 16 10:32:51.795762 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:51.795711    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:51.795954 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:51.795718    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:51.795954 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:51.795858    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:51.795954 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:51.795859    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:51.796055 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:51.795966    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:51.796055 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:51.796024    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:51.849969 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:51.849940    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lllwn"
Jul 16 10:32:51.850456 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:51.850440    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lllwn"
Jul 16 10:32:51.911104 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:51.911072    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lllwn"
Jul 16 10:32:51.911674 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:51.911465    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lllwn"
Jul 16 10:32:52.914936 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:52.914718    2576 generic.go:356] "Generic (PLEG): container finished" podID="1b89fcd0-9c09-475f-96b7-8f4029d612e2" containerID="919abd57ede249491b129d0aff196a0013ffdbd6ff038185b3704ba319d4987a" exitCode=0
Jul 16 10:32:52.915619 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:52.914807    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-drv8w" event={"ID":"1b89fcd0-9c09-475f-96b7-8f4029d612e2","Type":"ContainerDied","Data":"919abd57ede249491b129d0aff196a0013ffdbd6ff038185b3704ba319d4987a"}
Jul 16 10:32:52.918518 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:52.918499    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:32:52.918851 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:52.918830    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf" event={"ID":"5a80df04-e68b-404c-9ed7-96de4566f8a0","Type":"ContainerStarted","Data":"e26b075cc627190e189f8c28a8fcf1fe2bd22461214c3c27a6f4cd697959c797"}
Jul 16 10:32:52.919115 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:52.919093    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:52.919184 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:52.919177    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:52.919240 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:52.919212    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:52.919292 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:52.919277    2576 scope.go:117] "RemoveContainer" containerID="3a64a1bac682258b83b1e93d9930c697cfaf2cb466310e949e7059b879571abb"
Jul 16 10:32:52.935663 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:52.935645    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:52.935777 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:52.935766    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:32:53.795665 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:53.795585    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:53.795923 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:53.795585    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:53.795923 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:53.795700    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:53.795923 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:53.795598    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:53.795923 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:53.795797    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:53.795923 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:53.795826    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:53.924733 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:53.924703    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:32:53.925177 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:53.925158    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf" event={"ID":"5a80df04-e68b-404c-9ed7-96de4566f8a0","Type":"ContainerStarted","Data":"79dd6db7bd6e51620d44a7869b7cf098d4ab2a238e053f719c8676a0e3a49f11"}
Jul 16 10:32:53.957641 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:53.957594    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf" podStartSLOduration=8.851312363 podStartE2EDuration="27.957579713s" totalImagesPullingTime="19.10626735s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:53.956650373 +0000 UTC m=+28.693685645" watchObservedRunningTime="2026-07-16 10:32:53.957579713 +0000 UTC m=+28.694614983"
Jul 16 10:32:54.417233 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:54.417017    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r8z79"]
Jul 16 10:32:54.417457 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:54.417324    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:54.417457 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:54.417432    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:54.420024 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:54.419973    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pxkgn"]
Jul 16 10:32:54.420139 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:54.420074    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:54.420207 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:54.420167    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:54.420578 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:54.420557    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kq258"]
Jul 16 10:32:54.420646 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:54.420636    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:54.420734 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:54.420718    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:54.928943 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:54.928912    2576 generic.go:356] "Generic (PLEG): container finished" podID="1b89fcd0-9c09-475f-96b7-8f4029d612e2" containerID="c0f88ac18e2bdcd5ca7e25325221df728e826f0d2aa30c6435c03dc4c6634eba" exitCode=0
Jul 16 10:32:54.929667 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:54.928991    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-drv8w" event={"ID":"1b89fcd0-9c09-475f-96b7-8f4029d612e2","Type":"ContainerDied","Data":"c0f88ac18e2bdcd5ca7e25325221df728e826f0d2aa30c6435c03dc4c6634eba"}
Jul 16 10:32:55.796583 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:55.796548    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:55.796759 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:55.796638    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:55.796759 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:55.796668    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:55.796759 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:55.796714    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:56.795803 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:56.795712    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:56.796295 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:56.795853    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:56.935453 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:56.935422    2576 generic.go:356] "Generic (PLEG): container finished" podID="1b89fcd0-9c09-475f-96b7-8f4029d612e2" containerID="a7bef35f7f9e5ad6356e1bb1660944ad7284de974cc83964dd71be69ec185b43" exitCode=0
Jul 16 10:32:56.935605 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:56.935471    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-drv8w" event={"ID":"1b89fcd0-9c09-475f-96b7-8f4029d612e2","Type":"ContainerDied","Data":"a7bef35f7f9e5ad6356e1bb1660944ad7284de974cc83964dd71be69ec185b43"}
Jul 16 10:32:57.795632 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:57.795433    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:57.795828 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:57.795440    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:57.795828 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:57.795739    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:32:57.796257 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:57.795848    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:58.795164 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:58.795135    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:58.795334 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:58.795256    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pxkgn" podUID="78c8ab20-3864-469a-8af5-6e9778297bce"
Jul 16 10:32:59.456431 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:59.456384    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs\") pod \"network-metrics-daemon-kq258\" (UID: \"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f\") " pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:59.456431 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:59.456429    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:59.456891 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:59.456534    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:59.456891 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:59.456536    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:59.456891 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:59.456598    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret podName:30449683-8c30-47cf-a574-474beccc52b0 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:31.456579087 +0000 UTC m=+66.193614337 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret") pod "global-pull-secret-syncer-r8z79" (UID: "30449683-8c30-47cf-a574-474beccc52b0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 10:32:59.456891 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:59.456618    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs podName:0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f nodeName:}" failed. No retries permitted until 2026-07-16 10:33:31.45660805 +0000 UTC m=+66.193643301 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs") pod "network-metrics-daemon-kq258" (UID: "0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 10:32:59.557516 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:59.557473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb979\" (UniqueName: \"kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979\") pod \"network-check-target-pxkgn\" (UID: \"78c8ab20-3864-469a-8af5-6e9778297bce\") " pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:32:59.557700 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:59.557670    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 10:32:59.557700 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:59.557694    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 10:32:59.557843 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:59.557707    2576 projected.go:196] Error preparing data for projected volume kube-api-access-rb979 for pod openshift-network-diagnostics/network-check-target-pxkgn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:59.557843 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:59.557779    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979 podName:78c8ab20-3864-469a-8af5-6e9778297bce nodeName:}" failed. No retries permitted until 2026-07-16 10:33:31.557762149 +0000 UTC m=+66.294797414 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rb979" (UniqueName: "kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979") pod "network-check-target-pxkgn" (UID: "78c8ab20-3864-469a-8af5-6e9778297bce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 10:32:59.795201 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:59.795119    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:32:59.795342 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:59.795268    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kq258" podUID="0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f"
Jul 16 10:32:59.795342 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:32:59.795299    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:32:59.795459 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:32:59.795423    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r8z79" podUID="30449683-8c30-47cf-a574-474beccc52b0"
Jul 16 10:33:00.472092 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.472064    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-213.ec2.internal" event="NodeReady"
Jul 16 10:33:00.472532 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.472193    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 16 10:33:00.523057 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.523022    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bbs64"]
Jul 16 10:33:00.525456 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.525437    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:00.528287 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.528260    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 16 10:33:00.528603 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.528589    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 16 10:33:00.529245 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.529226    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x6pbd\""
Jul 16 10:33:00.532821 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.532785    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bb4br"]
Jul 16 10:33:00.534934 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.534918    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bb4br"
Jul 16 10:33:00.541569 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.541547    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 16 10:33:00.541652 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.541610    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 16 10:33:00.541969 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.541940    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-274x8\""
Jul 16 10:33:00.542192 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.542172    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 16 10:33:00.542980 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.542960    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bbs64"]
Jul 16 10:33:00.556946 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.556918    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bb4br"]
Jul 16 10:33:00.665853 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.665814    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m67q7\" (UniqueName: \"kubernetes.io/projected/fc917a38-428c-4044-bc83-21eb7e6b75ce-kube-api-access-m67q7\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:00.666041 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.665883    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41674660-ce10-48c5-a5d8-a57669fa42a8-cert\") pod \"ingress-canary-bb4br\" (UID: \"41674660-ce10-48c5-a5d8-a57669fa42a8\") " pod="openshift-ingress-canary/ingress-canary-bb4br"
Jul 16 10:33:00.666041 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.665925    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlmzk\" (UniqueName: \"kubernetes.io/projected/41674660-ce10-48c5-a5d8-a57669fa42a8-kube-api-access-dlmzk\") pod \"ingress-canary-bb4br\" (UID: \"41674660-ce10-48c5-a5d8-a57669fa42a8\") " pod="openshift-ingress-canary/ingress-canary-bb4br"
Jul 16 10:33:00.666041 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.665959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:00.666041 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.665985    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fc917a38-428c-4044-bc83-21eb7e6b75ce-config-volume\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:00.666041 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.666005    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc917a38-428c-4044-bc83-21eb7e6b75ce-tmp-dir\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:00.766997 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.766911    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlmzk\" (UniqueName: \"kubernetes.io/projected/41674660-ce10-48c5-a5d8-a57669fa42a8-kube-api-access-dlmzk\") pod \"ingress-canary-bb4br\" (UID: \"41674660-ce10-48c5-a5d8-a57669fa42a8\") " pod="openshift-ingress-canary/ingress-canary-bb4br"
Jul 16 10:33:00.766997 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.766973    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:00.767235 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.767009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fc917a38-428c-4044-bc83-21eb7e6b75ce-config-volume\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:00.767235 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.767036    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc917a38-428c-4044-bc83-21eb7e6b75ce-tmp-dir\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:00.767235 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:00.767076    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 10:33:00.767235 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:00.767144    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls podName:fc917a38-428c-4044-bc83-21eb7e6b75ce nodeName:}" failed. No retries permitted until 2026-07-16 10:33:01.267123126 +0000 UTC m=+36.004158383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls") pod "dns-default-bbs64" (UID: "fc917a38-428c-4044-bc83-21eb7e6b75ce") : secret "dns-default-metrics-tls" not found
Jul 16 10:33:00.767235 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.767057    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m67q7\" (UniqueName: \"kubernetes.io/projected/fc917a38-428c-4044-bc83-21eb7e6b75ce-kube-api-access-m67q7\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:00.767486 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.767288    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41674660-ce10-48c5-a5d8-a57669fa42a8-cert\") pod \"ingress-canary-bb4br\" (UID: \"41674660-ce10-48c5-a5d8-a57669fa42a8\") " pod="openshift-ingress-canary/ingress-canary-bb4br"
Jul 16 10:33:00.767569 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.767548    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc917a38-428c-4044-bc83-21eb7e6b75ce-tmp-dir\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:00.767801 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.767785    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fc917a38-428c-4044-bc83-21eb7e6b75ce-config-volume\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:00.771689 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.771664    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41674660-ce10-48c5-a5d8-a57669fa42a8-cert\") pod \"ingress-canary-bb4br\" (UID: \"41674660-ce10-48c5-a5d8-a57669fa42a8\") " pod="openshift-ingress-canary/ingress-canary-bb4br"
Jul 16 10:33:00.777694 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.777675    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m67q7\" (UniqueName: \"kubernetes.io/projected/fc917a38-428c-4044-bc83-21eb7e6b75ce-kube-api-access-m67q7\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:00.777904 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.777881    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlmzk\" (UniqueName: \"kubernetes.io/projected/41674660-ce10-48c5-a5d8-a57669fa42a8-kube-api-access-dlmzk\") pod \"ingress-canary-bb4br\" (UID: \"41674660-ce10-48c5-a5d8-a57669fa42a8\") " pod="openshift-ingress-canary/ingress-canary-bb4br"
Jul 16 10:33:00.795677 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.795657    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:33:00.798390 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.798367    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 10:33:00.798521 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.798503    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 10:33:00.799178 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.799160    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-42vxd\""
Jul 16 10:33:00.843894 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:00.843853    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bb4br"
Jul 16 10:33:01.270657 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:01.270625    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:01.270807 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:01.270760    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 10:33:01.270859 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:01.270817    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls podName:fc917a38-428c-4044-bc83-21eb7e6b75ce nodeName:}" failed. No retries permitted until 2026-07-16 10:33:02.270801793 +0000 UTC m=+37.007837047 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls") pod "dns-default-bbs64" (UID: "fc917a38-428c-4044-bc83-21eb7e6b75ce") : secret "dns-default-metrics-tls" not found
Jul 16 10:33:01.412306 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:01.412101    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bb4br"]
Jul 16 10:33:01.416720 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:01.416690    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41674660_ce10_48c5_a5d8_a57669fa42a8.slice/crio-9b330539114892a928a5a93fb7db975006096e6b34102b73ebe150103b88b37b WatchSource:0}: Error finding container 9b330539114892a928a5a93fb7db975006096e6b34102b73ebe150103b88b37b: Status 404 returned error can't find the container with id 9b330539114892a928a5a93fb7db975006096e6b34102b73ebe150103b88b37b
Jul 16 10:33:01.794971 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:01.794941    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:33:01.795401 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:01.795109    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:33:01.797713 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:01.797686    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 10:33:01.798485 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:01.798458    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 10:33:01.798485 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:01.798470    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xxmn5\""
Jul 16 10:33:01.952035 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:01.951995    2576 generic.go:356] "Generic (PLEG): container finished" podID="1b89fcd0-9c09-475f-96b7-8f4029d612e2" containerID="7c9e0d9346d91e307595113c82d7cf787fb0be0aa03337ef15ff0c8293e3383f" exitCode=0
Jul 16 10:33:01.952211 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:01.952064    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-drv8w" event={"ID":"1b89fcd0-9c09-475f-96b7-8f4029d612e2","Type":"ContainerDied","Data":"7c9e0d9346d91e307595113c82d7cf787fb0be0aa03337ef15ff0c8293e3383f"}
Jul 16 10:33:01.953310 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:01.953288    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bb4br" event={"ID":"41674660-ce10-48c5-a5d8-a57669fa42a8","Type":"ContainerStarted","Data":"9b330539114892a928a5a93fb7db975006096e6b34102b73ebe150103b88b37b"}
Jul 16 10:33:02.279081 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:02.279033    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:02.279238 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:02.279213    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 10:33:02.279292 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:02.279285    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls podName:fc917a38-428c-4044-bc83-21eb7e6b75ce nodeName:}" failed. No retries permitted until 2026-07-16 10:33:04.279266245 +0000 UTC m=+39.016301517 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls") pod "dns-default-bbs64" (UID: "fc917a38-428c-4044-bc83-21eb7e6b75ce") : secret "dns-default-metrics-tls" not found
Jul 16 10:33:02.959557 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:02.959522    2576 generic.go:356] "Generic (PLEG): container finished" podID="1b89fcd0-9c09-475f-96b7-8f4029d612e2" containerID="f1a9233246ec0b6d864bacd6ecc186d7e698f5b9b19a65e375f7d587d88f7c0a" exitCode=0
Jul 16 10:33:02.959996 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:02.959578    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-drv8w" event={"ID":"1b89fcd0-9c09-475f-96b7-8f4029d612e2","Type":"ContainerDied","Data":"f1a9233246ec0b6d864bacd6ecc186d7e698f5b9b19a65e375f7d587d88f7c0a"}
Jul 16 10:33:03.964638 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:03.964611    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-drv8w" event={"ID":"1b89fcd0-9c09-475f-96b7-8f4029d612e2","Type":"ContainerStarted","Data":"a45563e5505eb40db58d65221b18a29068d0622bbbbdbd8d560b09b5509b7eac"}
Jul 16 10:33:03.966020 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:03.965997    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bb4br" event={"ID":"41674660-ce10-48c5-a5d8-a57669fa42a8","Type":"ContainerStarted","Data":"ac2e5a6f5ae147c8f8e40f6036864cf8cd4a232fce8a697eefaee5014065e61c"}
Jul 16 10:33:03.995773 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:03.995663    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-drv8w" podStartSLOduration=8.132978762 podStartE2EDuration="37.995644933s" totalImagesPullingTime="29.862666171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:03.994627823 +0000 UTC m=+38.731663094" watchObservedRunningTime="2026-07-16 10:33:03.995644933 +0000 UTC m=+38.732680212"
Jul 16 10:33:04.024545 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:04.024472    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bb4br" podStartSLOduration=1.794671173 podStartE2EDuration="4.024455761s" totalImagesPullingTime="2.229784588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:04.02391934 +0000 UTC m=+38.760954614" watchObservedRunningTime="2026-07-16 10:33:04.024455761 +0000 UTC m=+38.761491075"
Jul 16 10:33:04.297034 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:04.296953    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:04.297180 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:04.297107    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 10:33:04.297180 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:04.297172    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls podName:fc917a38-428c-4044-bc83-21eb7e6b75ce nodeName:}" failed. No retries permitted until 2026-07-16 10:33:08.297156722 +0000 UTC m=+43.034192000 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls") pod "dns-default-bbs64" (UID: "fc917a38-428c-4044-bc83-21eb7e6b75ce") : secret "dns-default-metrics-tls" not found
Jul 16 10:33:06.747859 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:06.747825    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-z4vz5"]
Jul 16 10:33:06.786959 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:06.786918    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z4vz5"]
Jul 16 10:33:06.787114 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:06.786964    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:06.805265 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:06.805240    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 16 10:33:06.805387 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:06.805247    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8nb74\""
Jul 16 10:33:06.805387 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:06.805314    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 16 10:33:06.806272 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:06.806249    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 16 10:33:06.806359 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:06.806276    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 16 10:33:06.914159 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:06.914120    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m47cp\" (UniqueName: \"kubernetes.io/projected/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-api-access-m47cp\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:06.914325 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:06.914181    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2a5c84ae-5969-4224-b0e4-0afee0632152-crio-socket\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:06.914325 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:06.914199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a5c84ae-5969-4224-b0e4-0afee0632152-data-volume\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:06.914325 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:06.914256    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:06.914325 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:06.914315    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:07.015493 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.015414    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m47cp\" (UniqueName: \"kubernetes.io/projected/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-api-access-m47cp\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:07.015493 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.015463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2a5c84ae-5969-4224-b0e4-0afee0632152-crio-socket\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:07.015702 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.015568    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a5c84ae-5969-4224-b0e4-0afee0632152-data-volume\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:07.015702 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.015618    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2a5c84ae-5969-4224-b0e4-0afee0632152-crio-socket\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:07.015702 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.015618    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:07.015702 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.015671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:07.015917 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:07.015731    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 10:33:07.015917 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:07.015794    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 10:33:07.015917 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:07.015797    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm podName:2a5c84ae-5969-4224-b0e4-0afee0632152 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:07.515783338 +0000 UTC m=+42.252818588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z4vz5" (UID: "2a5c84ae-5969-4224-b0e4-0afee0632152") : configmap "kube-rbac-proxy" not found
Jul 16 10:33:07.015917 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:07.015855    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls podName:2a5c84ae-5969-4224-b0e4-0afee0632152 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:07.515841055 +0000 UTC m=+42.252876304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z4vz5" (UID: "2a5c84ae-5969-4224-b0e4-0afee0632152") : secret "insights-runtime-extractor-tls" not found
Jul 16 10:33:07.016084 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.015983    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a5c84ae-5969-4224-b0e4-0afee0632152-data-volume\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:07.025146 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.025126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m47cp\" (UniqueName: \"kubernetes.io/projected/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-api-access-m47cp\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:07.214050 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.214010    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5dmzd"]
Jul 16 10:33:07.236211 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.236183    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5dmzd"]
Jul 16 10:33:07.236352 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.236304    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5dmzd"
Jul 16 10:33:07.240596 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.240574    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 16 10:33:07.240731 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.240646    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 16 10:33:07.240731 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.240666    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mxdjp\""
Jul 16 10:33:07.318405 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.318325    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ln7c\" (UniqueName: \"kubernetes.io/projected/44bdc776-7b33-4be0-8ad7-827030065c04-kube-api-access-6ln7c\") pod \"migrator-7f59b54f6c-5dmzd\" (UID: \"44bdc776-7b33-4be0-8ad7-827030065c04\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5dmzd"
Jul 16 10:33:07.418905 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.418873    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ln7c\" (UniqueName: \"kubernetes.io/projected/44bdc776-7b33-4be0-8ad7-827030065c04-kube-api-access-6ln7c\") pod \"migrator-7f59b54f6c-5dmzd\" (UID: \"44bdc776-7b33-4be0-8ad7-827030065c04\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5dmzd"
Jul 16 10:33:07.428043 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.428021    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ln7c\" (UniqueName: \"kubernetes.io/projected/44bdc776-7b33-4be0-8ad7-827030065c04-kube-api-access-6ln7c\") pod \"migrator-7f59b54f6c-5dmzd\" (UID: \"44bdc776-7b33-4be0-8ad7-827030065c04\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5dmzd"
Jul 16 10:33:07.520153 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.520113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:07.520295 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.520170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:07.520295 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:07.520260    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 10:33:07.520376 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:07.520288    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 10:33:07.520376 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:07.520337    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm podName:2a5c84ae-5969-4224-b0e4-0afee0632152 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:08.520318145 +0000 UTC m=+43.257353417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z4vz5" (UID: "2a5c84ae-5969-4224-b0e4-0afee0632152") : configmap "kube-rbac-proxy" not found
Jul 16 10:33:07.520376 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:07.520352    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls podName:2a5c84ae-5969-4224-b0e4-0afee0632152 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:08.52034596 +0000 UTC m=+43.257381209 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z4vz5" (UID: "2a5c84ae-5969-4224-b0e4-0afee0632152") : secret "insights-runtime-extractor-tls" not found
Jul 16 10:33:07.545365 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.545338    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5dmzd"
Jul 16 10:33:07.706629 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.706591    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5dmzd"]
Jul 16 10:33:07.710425 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:07.710398    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44bdc776_7b33_4be0_8ad7_827030065c04.slice/crio-32c963e2767bc33209d76400ead565d47ee70dddd5b917eed8008b26832f27cd WatchSource:0}: Error finding container 32c963e2767bc33209d76400ead565d47ee70dddd5b917eed8008b26832f27cd: Status 404 returned error can't find the container with id 32c963e2767bc33209d76400ead565d47ee70dddd5b917eed8008b26832f27cd
Jul 16 10:33:07.975721 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:07.975687    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5dmzd" event={"ID":"44bdc776-7b33-4be0-8ad7-827030065c04","Type":"ContainerStarted","Data":"32c963e2767bc33209d76400ead565d47ee70dddd5b917eed8008b26832f27cd"}
Jul 16 10:33:08.326716 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:08.326634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:08.326888 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:08.326811    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 10:33:08.326888 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:08.326874    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls podName:fc917a38-428c-4044-bc83-21eb7e6b75ce nodeName:}" failed. No retries permitted until 2026-07-16 10:33:16.326859573 +0000 UTC m=+51.063894822 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls") pod "dns-default-bbs64" (UID: "fc917a38-428c-4044-bc83-21eb7e6b75ce") : secret "dns-default-metrics-tls" not found
Jul 16 10:33:08.528643 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:08.528613    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:08.528833 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:08.528674    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:08.528833 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:08.528779    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 10:33:08.528833 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:08.528798    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 10:33:08.528833 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:08.528852    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm podName:2a5c84ae-5969-4224-b0e4-0afee0632152 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:10.528828867 +0000 UTC m=+45.265864125 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z4vz5" (UID: "2a5c84ae-5969-4224-b0e4-0afee0632152") : configmap "kube-rbac-proxy" not found
Jul 16 10:33:08.528833 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:08.528872    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls podName:2a5c84ae-5969-4224-b0e4-0afee0632152 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:10.52886249 +0000 UTC m=+45.265897745 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z4vz5" (UID: "2a5c84ae-5969-4224-b0e4-0afee0632152") : secret "insights-runtime-extractor-tls" not found
Jul 16 10:33:08.537284 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:08.537263    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7l8m9_d3cc8c24-4def-40f0-9ee3-df7f26fdb61a/dns-node-resolver/0.log"
Jul 16 10:33:09.536778 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:09.536751    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d5zm6_2aeb2209-a85b-460e-a65c-cd36e86a5a8e/node-ca/0.log"
Jul 16 10:33:09.982141 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:09.982089    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5dmzd" event={"ID":"44bdc776-7b33-4be0-8ad7-827030065c04","Type":"ContainerStarted","Data":"88e3de35a429c19e7d7c6ccec623b7fa1ab932d75fcd6a4d8c43a4ad25461bb6"}
Jul 16 10:33:09.982310 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:09.982156    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5dmzd" event={"ID":"44bdc776-7b33-4be0-8ad7-827030065c04","Type":"ContainerStarted","Data":"32b1c7687e7da04dd9dd1d42dd02848cf9a5a543c79c7a93cf982aae0721318a"}
Jul 16 10:33:10.001756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:10.001694    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5dmzd" podStartSLOduration=1.155927796 podStartE2EDuration="3.001678938s" totalImagesPullingTime="1.845751142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:10.000817109 +0000 UTC m=+44.737852385" watchObservedRunningTime="2026-07-16 10:33:10.001678938 +0000 UTC m=+44.738714209"
Jul 16 10:33:10.541103 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:10.541067    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:10.541466 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:10.541115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:10.541466 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:10.541195    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 10:33:10.541466 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:10.541226    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 10:33:10.541466 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:10.541257    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm podName:2a5c84ae-5969-4224-b0e4-0afee0632152 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:14.541241335 +0000 UTC m=+49.278276606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z4vz5" (UID: "2a5c84ae-5969-4224-b0e4-0afee0632152") : configmap "kube-rbac-proxy" not found
Jul 16 10:33:10.541466 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:10.541280    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls podName:2a5c84ae-5969-4224-b0e4-0afee0632152 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:14.541266156 +0000 UTC m=+49.278301406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z4vz5" (UID: "2a5c84ae-5969-4224-b0e4-0afee0632152") : secret "insights-runtime-extractor-tls" not found
Jul 16 10:33:10.543028 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:10.543010    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bb4br_41674660-ce10-48c5-a5d8-a57669fa42a8/serve-healthcheck-canary/0.log"
Jul 16 10:33:14.573310 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:14.573272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:14.573897 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:14.573380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:14.573897 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:14.573518    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 10:33:14.573897 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:14.573587    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls podName:2a5c84ae-5969-4224-b0e4-0afee0632152 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:22.573568 +0000 UTC m=+57.310603263 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z4vz5" (UID: "2a5c84ae-5969-4224-b0e4-0afee0632152") : secret "insights-runtime-extractor-tls" not found
Jul 16 10:33:14.573897 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:14.573834    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2a5c84ae-5969-4224-b0e4-0afee0632152-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:16.388967 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:16.388935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:16.389331 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:16.389053    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 10:33:16.389331 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:16.389103    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls podName:fc917a38-428c-4044-bc83-21eb7e6b75ce nodeName:}" failed. No retries permitted until 2026-07-16 10:33:32.389090159 +0000 UTC m=+67.126125408 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls") pod "dns-default-bbs64" (UID: "fc917a38-428c-4044-bc83-21eb7e6b75ce") : secret "dns-default-metrics-tls" not found
Jul 16 10:33:22.638155 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:22.638113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:22.640385 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:22.640362    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2a5c84ae-5969-4224-b0e4-0afee0632152-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4vz5\" (UID: \"2a5c84ae-5969-4224-b0e4-0afee0632152\") " pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:22.695637 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:22.695604    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z4vz5"
Jul 16 10:33:22.840953 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:22.840919    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z4vz5"]
Jul 16 10:33:22.845232 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:22.845197    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a5c84ae_5969_4224_b0e4_0afee0632152.slice/crio-e5e0940d45ce92a500f3951c86de1687b4d20185e4140c51c7873214e76cb1eb WatchSource:0}: Error finding container e5e0940d45ce92a500f3951c86de1687b4d20185e4140c51c7873214e76cb1eb: Status 404 returned error can't find the container with id e5e0940d45ce92a500f3951c86de1687b4d20185e4140c51c7873214e76cb1eb
Jul 16 10:33:23.011795 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:23.011697    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z4vz5" event={"ID":"2a5c84ae-5969-4224-b0e4-0afee0632152","Type":"ContainerStarted","Data":"e5e0940d45ce92a500f3951c86de1687b4d20185e4140c51c7873214e76cb1eb"}
Jul 16 10:33:24.942802 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:24.942777    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7pfmf"
Jul 16 10:33:28.024679 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:28.024604    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z4vz5" event={"ID":"2a5c84ae-5969-4224-b0e4-0afee0632152","Type":"ContainerStarted","Data":"c83f234bf9c29140b627e6264c5a93112e840a2caaf3418bf39f33c5731c8fd0"}
Jul 16 10:33:31.034002 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.033970    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z4vz5" event={"ID":"2a5c84ae-5969-4224-b0e4-0afee0632152","Type":"ContainerStarted","Data":"ea011627f527e56093b84d72acac4754f6fbdd0c8013de38056abc89b5136df6"}
Jul 16 10:33:31.507513 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.507450    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs\") pod \"network-metrics-daemon-kq258\" (UID: \"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f\") " pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:33:31.507699 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.507531    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:33:31.510263 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.510234    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 10:33:31.510382 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.510299    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 10:33:31.520981 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.520955    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/30449683-8c30-47cf-a574-474beccc52b0-original-pull-secret\") pod \"global-pull-secret-syncer-r8z79\" (UID: \"30449683-8c30-47cf-a574-474beccc52b0\") " pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:33:31.521101 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.520991    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f-metrics-certs\") pod \"network-metrics-daemon-kq258\" (UID: \"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f\") " pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:33:31.608708 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.608672    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb979\" (UniqueName: \"kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979\") pod \"network-check-target-pxkgn\" (UID: \"78c8ab20-3864-469a-8af5-6e9778297bce\") " pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:33:31.611129 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.611107    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 10:33:31.621253 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.621231    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 10:33:31.632588 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.632566    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb979\" (UniqueName: \"kubernetes.io/projected/78c8ab20-3864-469a-8af5-6e9778297bce-kube-api-access-rb979\") pod \"network-check-target-pxkgn\" (UID: \"78c8ab20-3864-469a-8af5-6e9778297bce\") " pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:33:31.708830 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.708797    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-42vxd\""
Jul 16 10:33:31.717616 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.717585    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:33:31.807872 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.807847    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xxmn5\""
Jul 16 10:33:31.809940 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.809860    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r8z79"
Jul 16 10:33:31.816190 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.816110    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kq258"
Jul 16 10:33:31.857653 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.857623    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pxkgn"]
Jul 16 10:33:31.862628 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:31.862597    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c8ab20_3864_469a_8af5_6e9778297bce.slice/crio-561873dcfa7c122ceaf334cdf08269ed1f49b22e081a384c0bf88c389dc767ca WatchSource:0}: Error finding container 561873dcfa7c122ceaf334cdf08269ed1f49b22e081a384c0bf88c389dc767ca: Status 404 returned error can't find the container with id 561873dcfa7c122ceaf334cdf08269ed1f49b22e081a384c0bf88c389dc767ca
Jul 16 10:33:31.962354 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.962236    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r8z79"]
Jul 16 10:33:31.965388 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:31.965358    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30449683_8c30_47cf_a574_474beccc52b0.slice/crio-266db38ba58fad777fece500aa27727eef98dcc03f9c54bd5fa2fd411c260617 WatchSource:0}: Error finding container 266db38ba58fad777fece500aa27727eef98dcc03f9c54bd5fa2fd411c260617: Status 404 returned error can't find the container with id 266db38ba58fad777fece500aa27727eef98dcc03f9c54bd5fa2fd411c260617
Jul 16 10:33:31.981080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:31.981024    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kq258"]
Jul 16 10:33:31.984007 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:31.983979    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bf38c66_ddf8_4a4e_87bc_f66628fd6f5f.slice/crio-512da19c67eae852ad27d14180976e98b585bd26d9d6615d9ff391fea303c42d WatchSource:0}: Error finding container 512da19c67eae852ad27d14180976e98b585bd26d9d6615d9ff391fea303c42d: Status 404 returned error can't find the container with id 512da19c67eae852ad27d14180976e98b585bd26d9d6615d9ff391fea303c42d
Jul 16 10:33:32.038562 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.038457    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kq258" event={"ID":"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f","Type":"ContainerStarted","Data":"512da19c67eae852ad27d14180976e98b585bd26d9d6615d9ff391fea303c42d"}
Jul 16 10:33:32.039708 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.039686    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r8z79" event={"ID":"30449683-8c30-47cf-a574-474beccc52b0","Type":"ContainerStarted","Data":"266db38ba58fad777fece500aa27727eef98dcc03f9c54bd5fa2fd411c260617"}
Jul 16 10:33:32.040899 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.040878    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pxkgn" event={"ID":"78c8ab20-3864-469a-8af5-6e9778297bce","Type":"ContainerStarted","Data":"561873dcfa7c122ceaf334cdf08269ed1f49b22e081a384c0bf88c389dc767ca"}
Jul 16 10:33:32.414532 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.414487    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:32.417605 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.417549    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc917a38-428c-4044-bc83-21eb7e6b75ce-metrics-tls\") pod \"dns-default-bbs64\" (UID: \"fc917a38-428c-4044-bc83-21eb7e6b75ce\") " pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:32.432629 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.432598    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t"]
Jul 16 10:33:32.455630 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.455595    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wfnn6"]
Jul 16 10:33:32.455968 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.455946    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t"
Jul 16 10:33:32.460485 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.460465    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 16 10:33:32.460598 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.460499    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-pkm4m\""
Jul 16 10:33:32.460598 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.460551    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 16 10:33:32.460806 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.460788    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 16 10:33:32.460984 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.460969    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 16 10:33:32.471813 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.469882    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"]
Jul 16 10:33:32.471813 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.470065    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-wfnn6"
Jul 16 10:33:32.473220 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.473193    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-nmfr6\""
Jul 16 10:33:32.473599 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.473445    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 16 10:33:32.473681 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.473660    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 16 10:33:32.488989 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.488963    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wfnn6"]
Jul 16 10:33:32.489109 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.489009    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"]
Jul 16 10:33:32.489109 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.489031    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t"]
Jul 16 10:33:32.489223 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.489158    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"
Jul 16 10:33:32.495479 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.495426    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 16 10:33:32.515649 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.515618    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb-klusterlet-config\") pod \"klusterlet-addon-workmgr-774cf684c-5mw2b\" (UID: \"ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"
Jul 16 10:33:32.515802 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.515663    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6g7r\" (UniqueName: \"kubernetes.io/projected/ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb-kube-api-access-t6g7r\") pod \"klusterlet-addon-workmgr-774cf684c-5mw2b\" (UID: \"ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"
Jul 16 10:33:32.515802 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.515700    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t\" (UID: \"9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t"
Jul 16 10:33:32.515802 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.515730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c61920c1-33d8-43ae-b81e-be8e9b1cc13b-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wfnn6\" (UID: \"c61920c1-33d8-43ae-b81e-be8e9b1cc13b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wfnn6"
Jul 16 10:33:32.515802 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.515773    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c61920c1-33d8-43ae-b81e-be8e9b1cc13b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wfnn6\" (UID: \"c61920c1-33d8-43ae-b81e-be8e9b1cc13b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wfnn6"
Jul 16 10:33:32.515802 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.515800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb-tmp\") pod \"klusterlet-addon-workmgr-774cf684c-5mw2b\" (UID: \"ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"
Jul 16 10:33:32.516057 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.515825    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct7bp\" (UniqueName: \"kubernetes.io/projected/9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044-kube-api-access-ct7bp\") pod \"managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t\" (UID: \"9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t"
Jul 16 10:33:32.527323 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.527295    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-658996cf97-hss89"]
Jul 16 10:33:32.550157 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.548104    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.550157 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.549221    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-658996cf97-hss89"]
Jul 16 10:33:32.551076 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.551051    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 16 10:33:32.552793 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.552770    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 16 10:33:32.553085 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.553068    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9j5q6\""
Jul 16 10:33:32.553298 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.553280    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 16 10:33:32.559347 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.559328    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 16 10:33:32.616533 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d61ca23e-84d5-4e18-82b6-bfd2fc874869-registry-tls\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.616850 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616549    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c61920c1-33d8-43ae-b81e-be8e9b1cc13b-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wfnn6\" (UID: \"c61920c1-33d8-43ae-b81e-be8e9b1cc13b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wfnn6"
Jul 16 10:33:32.616850 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616582    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb-tmp\") pod \"klusterlet-addon-workmgr-774cf684c-5mw2b\" (UID: \"ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"
Jul 16 10:33:32.616850 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct7bp\" (UniqueName: \"kubernetes.io/projected/9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044-kube-api-access-ct7bp\") pod \"managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t\" (UID: \"9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t"
Jul 16 10:33:32.616850 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616642    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d61ca23e-84d5-4e18-82b6-bfd2fc874869-bound-sa-token\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.616850 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d61ca23e-84d5-4e18-82b6-bfd2fc874869-ca-trust-extracted\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.616850 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616729    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srl4k\" (UniqueName: \"kubernetes.io/projected/d61ca23e-84d5-4e18-82b6-bfd2fc874869-kube-api-access-srl4k\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.616850 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616774    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d61ca23e-84d5-4e18-82b6-bfd2fc874869-trusted-ca\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.616850 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616809    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d61ca23e-84d5-4e18-82b6-bfd2fc874869-image-registry-private-configuration\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.616850 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c61920c1-33d8-43ae-b81e-be8e9b1cc13b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wfnn6\" (UID: \"c61920c1-33d8-43ae-b81e-be8e9b1cc13b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wfnn6"
Jul 16 10:33:32.617450 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616871    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d61ca23e-84d5-4e18-82b6-bfd2fc874869-registry-certificates\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.617450 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616897    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d61ca23e-84d5-4e18-82b6-bfd2fc874869-installation-pull-secrets\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.617450 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t\" (UID: \"9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t"
Jul 16 10:33:32.617450 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb-klusterlet-config\") pod \"klusterlet-addon-workmgr-774cf684c-5mw2b\" (UID: \"ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"
Jul 16 10:33:32.617450 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.616994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6g7r\" (UniqueName: \"kubernetes.io/projected/ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb-kube-api-access-t6g7r\") pod \"klusterlet-addon-workmgr-774cf684c-5mw2b\" (UID: \"ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"
Jul 16 10:33:32.618097 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.618073    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c61920c1-33d8-43ae-b81e-be8e9b1cc13b-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wfnn6\" (UID: \"c61920c1-33d8-43ae-b81e-be8e9b1cc13b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wfnn6"
Jul 16 10:33:32.618398 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.618379    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb-tmp\") pod \"klusterlet-addon-workmgr-774cf684c-5mw2b\" (UID: \"ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"
Jul 16 10:33:32.625300 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.625272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb-klusterlet-config\") pod \"klusterlet-addon-workmgr-774cf684c-5mw2b\" (UID: \"ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"
Jul 16 10:33:32.625580 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.625535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t\" (UID: \"9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t"
Jul 16 10:33:32.626060 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.626033    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c61920c1-33d8-43ae-b81e-be8e9b1cc13b-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wfnn6\" (UID: \"c61920c1-33d8-43ae-b81e-be8e9b1cc13b\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wfnn6"
Jul 16 10:33:32.628314 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.628292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6g7r\" (UniqueName: \"kubernetes.io/projected/ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb-kube-api-access-t6g7r\") pod \"klusterlet-addon-workmgr-774cf684c-5mw2b\" (UID: \"ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"
Jul 16 10:33:32.630511 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.630469    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct7bp\" (UniqueName: \"kubernetes.io/projected/9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044-kube-api-access-ct7bp\") pod \"managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t\" (UID: \"9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t"
Jul 16 10:33:32.638405 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.638380    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x6pbd\""
Jul 16 10:33:32.646702 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.646442    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:32.718870 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.718040    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d61ca23e-84d5-4e18-82b6-bfd2fc874869-registry-tls\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.718870 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.718099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d61ca23e-84d5-4e18-82b6-bfd2fc874869-bound-sa-token\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.718870 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.718129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d61ca23e-84d5-4e18-82b6-bfd2fc874869-ca-trust-extracted\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.718870 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.718155    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srl4k\" (UniqueName: \"kubernetes.io/projected/d61ca23e-84d5-4e18-82b6-bfd2fc874869-kube-api-access-srl4k\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.718870 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.718191    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d61ca23e-84d5-4e18-82b6-bfd2fc874869-trusted-ca\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.718870 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.718224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d61ca23e-84d5-4e18-82b6-bfd2fc874869-image-registry-private-configuration\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.718870 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.718254    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d61ca23e-84d5-4e18-82b6-bfd2fc874869-registry-certificates\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.718870 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.718278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d61ca23e-84d5-4e18-82b6-bfd2fc874869-installation-pull-secrets\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.720001 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.719782    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d61ca23e-84d5-4e18-82b6-bfd2fc874869-ca-trust-extracted\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.721302 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.721256    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d61ca23e-84d5-4e18-82b6-bfd2fc874869-registry-certificates\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.722011 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.721851    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d61ca23e-84d5-4e18-82b6-bfd2fc874869-installation-pull-secrets\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.722179 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.722147    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d61ca23e-84d5-4e18-82b6-bfd2fc874869-trusted-ca\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.722241 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.722154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d61ca23e-84d5-4e18-82b6-bfd2fc874869-image-registry-private-configuration\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.723529 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.723468    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d61ca23e-84d5-4e18-82b6-bfd2fc874869-registry-tls\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.732836 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.732808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srl4k\" (UniqueName: \"kubernetes.io/projected/d61ca23e-84d5-4e18-82b6-bfd2fc874869-kube-api-access-srl4k\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.734789 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.734765    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d61ca23e-84d5-4e18-82b6-bfd2fc874869-bound-sa-token\") pod \"image-registry-658996cf97-hss89\" (UID: \"d61ca23e-84d5-4e18-82b6-bfd2fc874869\") " pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:32.777207 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.776875    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t"
Jul 16 10:33:32.787228 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.787197    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-wfnn6"
Jul 16 10:33:32.805828 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.805798    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"
Jul 16 10:33:32.851209 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.851179    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bbs64"]
Jul 16 10:33:32.860029 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:32.859974    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc917a38_428c_4044_bc83_21eb7e6b75ce.slice/crio-c5046259d49bc5c268e404c259d3413148a9efeb70afdb3a5af8d1055c70375f WatchSource:0}: Error finding container c5046259d49bc5c268e404c259d3413148a9efeb70afdb3a5af8d1055c70375f: Status 404 returned error can't find the container with id c5046259d49bc5c268e404c259d3413148a9efeb70afdb3a5af8d1055c70375f
Jul 16 10:33:32.861290 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:32.861225    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:33.023257 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:33.023023    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t"]
Jul 16 10:33:33.046729 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:33.046658    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wfnn6"]
Jul 16 10:33:33.050175 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:33.050139    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bbs64" event={"ID":"fc917a38-428c-4044-bc83-21eb7e6b75ce","Type":"ContainerStarted","Data":"c5046259d49bc5c268e404c259d3413148a9efeb70afdb3a5af8d1055c70375f"}
Jul 16 10:33:33.081650 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:33.081574    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"]
Jul 16 10:33:33.112657 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:33.112625    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-658996cf97-hss89"]
Jul 16 10:33:33.380170 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:33.379605    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd61ca23e_84d5_4e18_82b6_bfd2fc874869.slice/crio-6b9f0291212ed5057ea5c3db36470aee05f4400fd6ba375fbddf577208ab9d80 WatchSource:0}: Error finding container 6b9f0291212ed5057ea5c3db36470aee05f4400fd6ba375fbddf577208ab9d80: Status 404 returned error can't find the container with id 6b9f0291212ed5057ea5c3db36470aee05f4400fd6ba375fbddf577208ab9d80
Jul 16 10:33:33.381058 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:33.380975    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c97cb0d_ffa0_4a68_aea0_25d3f5fc3044.slice/crio-9eafc4de453aec572d29504cb260e97c33a18150273caaee631fecc7579f3237 WatchSource:0}: Error finding container 9eafc4de453aec572d29504cb260e97c33a18150273caaee631fecc7579f3237: Status 404 returned error can't find the container with id 9eafc4de453aec572d29504cb260e97c33a18150273caaee631fecc7579f3237
Jul 16 10:33:33.382021 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:33.381571    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61920c1_33d8_43ae_b81e_be8e9b1cc13b.slice/crio-1a346bfff639964a72ebdc1c9a3f418868d378354eb1ab965928d8b8f401499f WatchSource:0}: Error finding container 1a346bfff639964a72ebdc1c9a3f418868d378354eb1ab965928d8b8f401499f: Status 404 returned error can't find the container with id 1a346bfff639964a72ebdc1c9a3f418868d378354eb1ab965928d8b8f401499f
Jul 16 10:33:33.383121 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:33.383094    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce3fbc6b_cfb4_4cb5_bea7_b0c268861bcb.slice/crio-29cad8c4512331a106a8d681afe390957606820a31c991ae974c0825a8cb9af4 WatchSource:0}: Error finding container 29cad8c4512331a106a8d681afe390957606820a31c991ae974c0825a8cb9af4: Status 404 returned error can't find the container with id 29cad8c4512331a106a8d681afe390957606820a31c991ae974c0825a8cb9af4
Jul 16 10:33:34.055585 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:34.055507    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t" event={"ID":"9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044","Type":"ContainerStarted","Data":"9eafc4de453aec572d29504cb260e97c33a18150273caaee631fecc7579f3237"}
Jul 16 10:33:34.058350 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:34.058268    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-wfnn6" event={"ID":"c61920c1-33d8-43ae-b81e-be8e9b1cc13b","Type":"ContainerStarted","Data":"1a346bfff639964a72ebdc1c9a3f418868d378354eb1ab965928d8b8f401499f"}
Jul 16 10:33:34.060539 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:34.060513    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-658996cf97-hss89" event={"ID":"d61ca23e-84d5-4e18-82b6-bfd2fc874869","Type":"ContainerStarted","Data":"6b9f0291212ed5057ea5c3db36470aee05f4400fd6ba375fbddf577208ab9d80"}
Jul 16 10:33:34.062065 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:34.061985    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b" event={"ID":"ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb","Type":"ContainerStarted","Data":"29cad8c4512331a106a8d681afe390957606820a31c991ae974c0825a8cb9af4"}
Jul 16 10:33:39.649709 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:39.649678    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl"]
Jul 16 10:33:39.656199 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:39.656167    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl"
Jul 16 10:33:39.658558 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:39.658536    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 16 10:33:39.658690 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:39.658612    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-9wzmt\""
Jul 16 10:33:39.665325 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:39.665300    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl"]
Jul 16 10:33:39.774635 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:39.774594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f3d2592e-cef7-45c0-a246-e98739f3413a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-pxxgl\" (UID: \"f3d2592e-cef7-45c0-a246-e98739f3413a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl"
Jul 16 10:33:39.875638 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:39.875596    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f3d2592e-cef7-45c0-a246-e98739f3413a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-pxxgl\" (UID: \"f3d2592e-cef7-45c0-a246-e98739f3413a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl"
Jul 16 10:33:39.875819 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:39.875798    2576 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 16 10:33:39.875889 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:33:39.875878    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d2592e-cef7-45c0-a246-e98739f3413a-tls-certificates podName:f3d2592e-cef7-45c0-a246-e98739f3413a nodeName:}" failed. No retries permitted until 2026-07-16 10:33:40.375860865 +0000 UTC m=+75.112896115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/f3d2592e-cef7-45c0-a246-e98739f3413a-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-pxxgl" (UID: "f3d2592e-cef7-45c0-a246-e98739f3413a") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 16 10:33:40.379667 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:40.379626    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f3d2592e-cef7-45c0-a246-e98739f3413a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-pxxgl\" (UID: \"f3d2592e-cef7-45c0-a246-e98739f3413a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl"
Jul 16 10:33:40.382638 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:40.382586    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f3d2592e-cef7-45c0-a246-e98739f3413a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-pxxgl\" (UID: \"f3d2592e-cef7-45c0-a246-e98739f3413a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl"
Jul 16 10:33:40.567255 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:40.567168    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl"
Jul 16 10:33:41.527436 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:41.527395    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl"]
Jul 16 10:33:41.534081 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:41.534045    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d2592e_cef7_45c0_a246_e98739f3413a.slice/crio-bb047c1d7adbb2a5f6c20da0c42f1da56c7cb521ce2530a67a8bb0d3d75d1519 WatchSource:0}: Error finding container bb047c1d7adbb2a5f6c20da0c42f1da56c7cb521ce2530a67a8bb0d3d75d1519: Status 404 returned error can't find the container with id bb047c1d7adbb2a5f6c20da0c42f1da56c7cb521ce2530a67a8bb0d3d75d1519
Jul 16 10:33:42.090682 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.090585    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t" event={"ID":"9c97cb0d-ffa0-4a68-aea0-25d3f5fc3044","Type":"ContainerStarted","Data":"af414cb3e3bff774179e5789a8ddedccbe9c3cdb19bfe4ea72560100f38d0374"}
Jul 16 10:33:42.091762 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.091721    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl" event={"ID":"f3d2592e-cef7-45c0-a246-e98739f3413a","Type":"ContainerStarted","Data":"bb047c1d7adbb2a5f6c20da0c42f1da56c7cb521ce2530a67a8bb0d3d75d1519"}
Jul 16 10:33:42.093593 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.093573    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z4vz5" event={"ID":"2a5c84ae-5969-4224-b0e4-0afee0632152","Type":"ContainerStarted","Data":"af9e0ea608a4c6bb357e74403cdf058b0d3f8a58bf8dd681a4fca23771eebf2a"}
Jul 16 10:33:42.095212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.095194    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kq258" event={"ID":"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f","Type":"ContainerStarted","Data":"03fea0a1fbc82a9a6974493c36e90489f3194f77a372c618733e827fa6ebd011"}
Jul 16 10:33:42.095308 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.095230    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kq258" event={"ID":"0bf38c66-ddf8-4a4e-87bc-f66628fd6f5f","Type":"ContainerStarted","Data":"46c83ba359b0ba86a70cb3a07ba0abc281bfe83768c17287b3b2f532f3400429"}
Jul 16 10:33:42.096656 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.096636    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bbs64" event={"ID":"fc917a38-428c-4044-bc83-21eb7e6b75ce","Type":"ContainerStarted","Data":"86e4457f78329e139e3b20e094c6264daca24626713cf9d7e8a93db7a0add517"}
Jul 16 10:33:42.096771 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.096676    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bbs64" event={"ID":"fc917a38-428c-4044-bc83-21eb7e6b75ce","Type":"ContainerStarted","Data":"057e0237d64e85d534b08ab83ab78a1eef0639cc10c7fc2e2ad64ca9fc26c6aa"}
Jul 16 10:33:42.096771 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.096727    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:42.098033 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.098013    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-wfnn6" event={"ID":"c61920c1-33d8-43ae-b81e-be8e9b1cc13b","Type":"ContainerStarted","Data":"7e8dd52da8ee2779383dba8e5a82dbe444855c3f02b76133b37dcda43102413a"}
Jul 16 10:33:42.099371 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.099345    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r8z79" event={"ID":"30449683-8c30-47cf-a574-474beccc52b0","Type":"ContainerStarted","Data":"1f5adcc4ffd6f1d98f4a51306706204f0f34884ad4cfae436d9eac5b6cc77379"}
Jul 16 10:33:42.100591 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.100563    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pxkgn" event={"ID":"78c8ab20-3864-469a-8af5-6e9778297bce","Type":"ContainerStarted","Data":"b28c3e59bcf12a046adceeb08b53ceaa73621bf3cc40a344f4127d49b5df8ef2"}
Jul 16 10:33:42.100669 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.100606    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:33:42.105585 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.105552    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-658996cf97-hss89" event={"ID":"d61ca23e-84d5-4e18-82b6-bfd2fc874869","Type":"ContainerStarted","Data":"c0ce461a97c1dd2245a1de8f514f7584ea22958c3ce944a3ebe25935be2fc23f"}
Jul 16 10:33:42.105685 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.105628    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:33:42.106894 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.106864    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b" event={"ID":"ce3fbc6b-cfb4-4cb5-bea7-b0c268861bcb","Type":"ContainerStarted","Data":"1381932a92573c07a6751422a3d1cb5330ebd5466a71207da9b24086286752da"}
Jul 16 10:33:42.107145 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.107049    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"
Jul 16 10:33:42.108349 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.108237    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-699f6d6cbd-jsr6t" podStartSLOduration=2.128477709 podStartE2EDuration="10.108221585s" totalImagesPullingTime="7.979743876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:42.107477949 +0000 UTC m=+76.844513217" watchObservedRunningTime="2026-07-16 10:33:42.108221585 +0000 UTC m=+76.845256834"
Jul 16 10:33:42.108902 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.108884    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b"
Jul 16 10:33:42.129961 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.129918    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-774cf684c-5mw2b" podStartSLOduration=2.127554716 podStartE2EDuration="10.129903437s" totalImagesPullingTime="8.002348721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:42.129359928 +0000 UTC m=+76.866395202" watchObservedRunningTime="2026-07-16 10:33:42.129903437 +0000 UTC m=+76.866938706"
Jul 16 10:33:42.151043 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.150994    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bbs64" podStartSLOduration=33.656234769 podStartE2EDuration="42.150981746s" totalImagesPullingTime="8.494746977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:42.150225827 +0000 UTC m=+76.887261100" watchObservedRunningTime="2026-07-16 10:33:42.150981746 +0000 UTC m=+76.888016994"
Jul 16 10:33:42.175037 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.174993    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-658996cf97-hss89" podStartSLOduration=10.174977785 podStartE2EDuration="10.174977785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:42.174034256 +0000 UTC m=+76.911069528" watchObservedRunningTime="2026-07-16 10:33:42.174977785 +0000 UTC m=+76.912013055"
Jul 16 10:33:42.193042 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.192989    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kq258" podStartSLOduration=68.012034935 podStartE2EDuration="1m16.192970475s" totalImagesPullingTime="8.18093554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:42.192352406 +0000 UTC m=+76.929387674" watchObservedRunningTime="2026-07-16 10:33:42.192970475 +0000 UTC m=+76.930005747"
Jul 16 10:33:42.218094 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.218042    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-z4vz5" podStartSLOduration=19.313058243 podStartE2EDuration="36.218022441s" totalImagesPullingTime="16.904964198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:33:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:42.217264803 +0000 UTC m=+76.954300084" watchObservedRunningTime="2026-07-16 10:33:42.218022441 +0000 UTC m=+76.955057713"
Jul 16 10:33:42.237461 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.237415    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-r8z79" podStartSLOduration=68.360425027 podStartE2EDuration="1m16.237399781s" totalImagesPullingTime="7.876974754s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:42.234987901 +0000 UTC m=+76.972023174" watchObservedRunningTime="2026-07-16 10:33:42.237399781 +0000 UTC m=+76.974435052"
Jul 16 10:33:42.278760 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.278684    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pxkgn" podStartSLOduration=67.976218101 podStartE2EDuration="1m16.278667965s" totalImagesPullingTime="8.302449864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:42.252354408 +0000 UTC m=+76.989389690" watchObservedRunningTime="2026-07-16 10:33:42.278667965 +0000 UTC m=+77.015703238"
Jul 16 10:33:42.279711 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:42.279668    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-wfnn6" podStartSLOduration=2.311681174 podStartE2EDuration="10.279658631s" totalImagesPullingTime="7.967977457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:42.278736358 +0000 UTC m=+77.015771629" watchObservedRunningTime="2026-07-16 10:33:42.279658631 +0000 UTC m=+77.016693901"
Jul 16 10:33:43.112269 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.112232    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl" event={"ID":"f3d2592e-cef7-45c0-a246-e98739f3413a","Type":"ContainerStarted","Data":"b9c13db44d57ea0ab3558420f23ac9fc5e9d1d3a9c52fa908d8461c0ee0a8aca"}
Jul 16 10:33:43.112769 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.112489    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl"
Jul 16 10:33:43.117265 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.117242    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl"
Jul 16 10:33:43.138495 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.138437    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pxxgl" podStartSLOduration=2.87880883 podStartE2EDuration="4.138423497s" totalImagesPullingTime="1.259614667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:43.136714246 +0000 UTC m=+77.873749517" watchObservedRunningTime="2026-07-16 10:33:43.138423497 +0000 UTC m=+77.875458767"
Jul 16 10:33:43.781413 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.781380    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-nngw4"]
Jul 16 10:33:43.784466 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.784451    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:43.789281 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.789260    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 16 10:33:43.790251 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.790169    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 16 10:33:43.790251 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.790224    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 16 10:33:43.790484 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.790467    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 16 10:33:43.790781 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.790760    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 16 10:33:43.791089 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.791074    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-r7tnk\""
Jul 16 10:33:43.806788 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.804349    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-nngw4"]
Jul 16 10:33:43.904667 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.904629    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bda89000-0ee6-41f4-a41e-631d65d020dc-metrics-client-ca\") pod \"prometheus-operator-64785b6675-nngw4\" (UID: \"bda89000-0ee6-41f4-a41e-631d65d020dc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:43.904667 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.904670    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bda89000-0ee6-41f4-a41e-631d65d020dc-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-nngw4\" (UID: \"bda89000-0ee6-41f4-a41e-631d65d020dc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:43.904900 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.904701    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bda89000-0ee6-41f4-a41e-631d65d020dc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-nngw4\" (UID: \"bda89000-0ee6-41f4-a41e-631d65d020dc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:43.904900 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:43.904730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9qp6\" (UniqueName: \"kubernetes.io/projected/bda89000-0ee6-41f4-a41e-631d65d020dc-kube-api-access-j9qp6\") pod \"prometheus-operator-64785b6675-nngw4\" (UID: \"bda89000-0ee6-41f4-a41e-631d65d020dc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:44.005474 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:44.005439    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bda89000-0ee6-41f4-a41e-631d65d020dc-metrics-client-ca\") pod \"prometheus-operator-64785b6675-nngw4\" (UID: \"bda89000-0ee6-41f4-a41e-631d65d020dc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:44.005578 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:44.005477    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bda89000-0ee6-41f4-a41e-631d65d020dc-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-nngw4\" (UID: \"bda89000-0ee6-41f4-a41e-631d65d020dc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:44.005578 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:44.005516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bda89000-0ee6-41f4-a41e-631d65d020dc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-nngw4\" (UID: \"bda89000-0ee6-41f4-a41e-631d65d020dc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:44.005578 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:44.005553    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9qp6\" (UniqueName: \"kubernetes.io/projected/bda89000-0ee6-41f4-a41e-631d65d020dc-kube-api-access-j9qp6\") pod \"prometheus-operator-64785b6675-nngw4\" (UID: \"bda89000-0ee6-41f4-a41e-631d65d020dc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:44.006211 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:44.006187    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bda89000-0ee6-41f4-a41e-631d65d020dc-metrics-client-ca\") pod \"prometheus-operator-64785b6675-nngw4\" (UID: \"bda89000-0ee6-41f4-a41e-631d65d020dc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:44.007965 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:44.007938    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bda89000-0ee6-41f4-a41e-631d65d020dc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-nngw4\" (UID: \"bda89000-0ee6-41f4-a41e-631d65d020dc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:44.008045 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:44.008024    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bda89000-0ee6-41f4-a41e-631d65d020dc-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-nngw4\" (UID: \"bda89000-0ee6-41f4-a41e-631d65d020dc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:44.016139 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:44.016108    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9qp6\" (UniqueName: \"kubernetes.io/projected/bda89000-0ee6-41f4-a41e-631d65d020dc-kube-api-access-j9qp6\") pod \"prometheus-operator-64785b6675-nngw4\" (UID: \"bda89000-0ee6-41f4-a41e-631d65d020dc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:44.093316 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:44.093223    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4"
Jul 16 10:33:44.218792 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:44.218761    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-nngw4"]
Jul 16 10:33:44.222154 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:44.222132    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda89000_0ee6_41f4_a41e_631d65d020dc.slice/crio-c0aa2d10032d5951becd93f968e6b9254fa40d30c5b61d7551fe7e80ac99ca7c WatchSource:0}: Error finding container c0aa2d10032d5951becd93f968e6b9254fa40d30c5b61d7551fe7e80ac99ca7c: Status 404 returned error can't find the container with id c0aa2d10032d5951becd93f968e6b9254fa40d30c5b61d7551fe7e80ac99ca7c
Jul 16 10:33:45.121134 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:45.121102    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4" event={"ID":"bda89000-0ee6-41f4-a41e-631d65d020dc","Type":"ContainerStarted","Data":"c0aa2d10032d5951becd93f968e6b9254fa40d30c5b61d7551fe7e80ac99ca7c"}
Jul 16 10:33:46.128721 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:46.128679    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4" event={"ID":"bda89000-0ee6-41f4-a41e-631d65d020dc","Type":"ContainerStarted","Data":"8d2a919ce3b9fbadd8979fcb515d36af159c74c5bb6a44f1490432e2fb563dd7"}
Jul 16 10:33:46.129223 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:46.128772    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4" event={"ID":"bda89000-0ee6-41f4-a41e-631d65d020dc","Type":"ContainerStarted","Data":"cdf7d415fc92e76e588c4c839f9916edb38df0d3f074e6f5d80c40c63a531880"}
Jul 16 10:33:46.153153 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:46.153098    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-nngw4" podStartSLOduration=1.579718086 podStartE2EDuration="3.153080246s" totalImagesPullingTime="1.57336216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:46.15202445 +0000 UTC m=+80.889059722" watchObservedRunningTime="2026-07-16 10:33:46.153080246 +0000 UTC m=+80.890115516"
Jul 16 10:33:48.508963 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.508830    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6n659"]
Jul 16 10:33:48.512242 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.512220    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.526546 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.526519    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 16 10:33:48.527406 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.527386    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8gj4m\""
Jul 16 10:33:48.527994 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.527980    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 16 10:33:48.541996 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.541975    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-accelerators-collector-config\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.542064 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.542010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-root\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.542064 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.542056    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-metrics-client-ca\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.542130 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.542074    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.542130 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.542089    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-wtmp\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.542191 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.542138    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-tls\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.542191 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.542163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcmp9\" (UniqueName: \"kubernetes.io/projected/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-kube-api-access-xcmp9\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.542191 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.542185    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-sys\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.542277 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.542211    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-textfile\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.548439 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.548421    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 16 10:33:48.642987 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.642955    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-accelerators-collector-config\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643177 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.642995    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-root\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643177 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.643029    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-metrics-client-ca\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643177 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.643051    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643177 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.643066    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-wtmp\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643177 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.643083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-tls\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643177 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.643082    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-root\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643177 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.643100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcmp9\" (UniqueName: \"kubernetes.io/projected/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-kube-api-access-xcmp9\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643177 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.643142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-sys\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643177 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.643172    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-textfile\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643607 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.643236    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-wtmp\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643607 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.643274    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-sys\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643607 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.643550    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-textfile\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643703 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.643675    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-accelerators-collector-config\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.643703 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.643686    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-metrics-client-ca\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.645484 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.645468    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-tls\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.645543 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.645512    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.682504 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.682467    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcmp9\" (UniqueName: \"kubernetes.io/projected/1c84770f-b939-48e8-a3fa-61f1d5fcccb5-kube-api-access-xcmp9\") pod \"node-exporter-6n659\" (UID: \"1c84770f-b939-48e8-a3fa-61f1d5fcccb5\") " pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.821128 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:48.821027    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6n659"
Jul 16 10:33:48.831132 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:48.831101    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c84770f_b939_48e8_a3fa_61f1d5fcccb5.slice/crio-b372180621413d92dbf244ca37b260894f3f0753b16da77d15722e199ebb45a2 WatchSource:0}: Error finding container b372180621413d92dbf244ca37b260894f3f0753b16da77d15722e199ebb45a2: Status 404 returned error can't find the container with id b372180621413d92dbf244ca37b260894f3f0753b16da77d15722e199ebb45a2
Jul 16 10:33:49.143684 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:49.143652    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6n659" event={"ID":"1c84770f-b939-48e8-a3fa-61f1d5fcccb5","Type":"ContainerStarted","Data":"b372180621413d92dbf244ca37b260894f3f0753b16da77d15722e199ebb45a2"}
Jul 16 10:33:50.148249 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:50.148213    2576 generic.go:356] "Generic (PLEG): container finished" podID="1c84770f-b939-48e8-a3fa-61f1d5fcccb5" containerID="3b90742faf193132d7baaeb741586e90f28c41aac8fe8edbbdd335fee26e28e2" exitCode=0
Jul 16 10:33:50.148640 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:50.148287    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6n659" event={"ID":"1c84770f-b939-48e8-a3fa-61f1d5fcccb5","Type":"ContainerDied","Data":"3b90742faf193132d7baaeb741586e90f28c41aac8fe8edbbdd335fee26e28e2"}
Jul 16 10:33:51.154331 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.154285    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6n659" event={"ID":"1c84770f-b939-48e8-a3fa-61f1d5fcccb5","Type":"ContainerStarted","Data":"a0f689418306d90a6924e687cf5b71ec23ef0345b5f5bddcd261703ad820f257"}
Jul 16 10:33:51.154697 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.154402    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6n659" event={"ID":"1c84770f-b939-48e8-a3fa-61f1d5fcccb5","Type":"ContainerStarted","Data":"b9f7bcef6253ae5812e25c5c6ceec840dc4c2e49a1ad44de71ee9f27f940db63"}
Jul 16 10:33:51.186212 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.186164    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6n659" podStartSLOduration=2.248315503 podStartE2EDuration="3.186150008s" totalImagesPullingTime="874.168411ms" totalInitContainerRuntime="63.666094ms" isStatefulPod=true podCreationTimestamp="2026-07-16 10:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:51.185949707 +0000 UTC m=+85.922984978" watchObservedRunningTime="2026-07-16 10:33:51.186150008 +0000 UTC m=+85.923185278"
Jul 16 10:33:51.298092 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.298059    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"]
Jul 16 10:33:51.300579 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.300565    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.303185 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.303160    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 16 10:33:51.303355 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.303334    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 16 10:33:51.303454 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.303439    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 16 10:33:51.303500 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.303473    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-j97qh\""
Jul 16 10:33:51.303734 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.303718    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 16 10:33:51.303837 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.303818    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-35irjh7tce27g\""
Jul 16 10:33:51.303917 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.303903    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 16 10:33:51.321798 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.321771    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"]
Jul 16 10:33:51.364145 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.364103    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-grpc-tls\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.364145 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.364151    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.364377 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.364208    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-metrics-client-ca\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.364377 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.364258    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-tls\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.364377 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.364279    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q44k5\" (UniqueName: \"kubernetes.io/projected/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-kube-api-access-q44k5\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.364377 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.364329    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.364377 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.364353    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.364377 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.364371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.464998 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.464911    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.464998 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.464956    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-metrics-client-ca\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.464998 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.464994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-tls\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.465262 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.465017    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q44k5\" (UniqueName: \"kubernetes.io/projected/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-kube-api-access-q44k5\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.465262 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.465063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.465262 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.465151    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.465262 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.465190    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.465262 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.465223    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-grpc-tls\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.465867 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.465843    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-metrics-client-ca\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.467849 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.467822    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.467960 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.467831    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-tls\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.468039 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.468021    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.468039 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.468033    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.468219 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.468206    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-grpc-tls\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.468328 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.468311    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.473591 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.473571    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q44k5\" (UniqueName: \"kubernetes.io/projected/a6454579-1b08-4ca5-bd1a-94284b0d0ecb-kube-api-access-q44k5\") pod \"thanos-querier-c9b88f75f-8l7tc\" (UID: \"a6454579-1b08-4ca5-bd1a-94284b0d0ecb\") " pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.610114 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.610082    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:51.754339 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:51.754257    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"]
Jul 16 10:33:51.757955 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:51.757931    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6454579_1b08_4ca5_bd1a_94284b0d0ecb.slice/crio-af332fdd52b7ed55748d65af0d2f1e44ed0569779cc82a7e7e75ac5e307cb901 WatchSource:0}: Error finding container af332fdd52b7ed55748d65af0d2f1e44ed0569779cc82a7e7e75ac5e307cb901: Status 404 returned error can't find the container with id af332fdd52b7ed55748d65af0d2f1e44ed0569779cc82a7e7e75ac5e307cb901
Jul 16 10:33:52.115019 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:52.114990    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bbs64"
Jul 16 10:33:52.160039 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:52.160007    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc" event={"ID":"a6454579-1b08-4ca5-bd1a-94284b0d0ecb","Type":"ContainerStarted","Data":"af332fdd52b7ed55748d65af0d2f1e44ed0569779cc82a7e7e75ac5e307cb901"}
Jul 16 10:33:53.233639 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.233608    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-db5bc955b-67wg2"]
Jul 16 10:33:53.236203 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.236185    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.240014 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.239996    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 16 10:33:53.241049 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.241028    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 16 10:33:53.245004 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.244982    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-c8msr\""
Jul 16 10:33:53.245105 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.245075    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 16 10:33:53.245796 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.245777    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 16 10:33:53.246362 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.246343    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-smadqhtgn683\""
Jul 16 10:33:53.249989 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.249971    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-db5bc955b-67wg2"]
Jul 16 10:33:53.279220 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.279184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63191a58-2211-47a0-979c-f8f6821e540a-client-ca-bundle\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.284268 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.280013    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/63191a58-2211-47a0-979c-f8f6821e540a-audit-log\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.284268 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.280086    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqrgd\" (UniqueName: \"kubernetes.io/projected/63191a58-2211-47a0-979c-f8f6821e540a-kube-api-access-fqrgd\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.284268 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.280375    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/63191a58-2211-47a0-979c-f8f6821e540a-secret-metrics-server-client-certs\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.284268 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.280437    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/63191a58-2211-47a0-979c-f8f6821e540a-secret-metrics-server-tls\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.284268 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.280477    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63191a58-2211-47a0-979c-f8f6821e540a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.284268 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.280527    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/63191a58-2211-47a0-979c-f8f6821e540a-metrics-server-audit-profiles\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.381469 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.381432    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63191a58-2211-47a0-979c-f8f6821e540a-client-ca-bundle\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.381657 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.381487    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/63191a58-2211-47a0-979c-f8f6821e540a-audit-log\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.381657 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.381518    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqrgd\" (UniqueName: \"kubernetes.io/projected/63191a58-2211-47a0-979c-f8f6821e540a-kube-api-access-fqrgd\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.381657 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.381554    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/63191a58-2211-47a0-979c-f8f6821e540a-secret-metrics-server-client-certs\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.381657 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.381576    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/63191a58-2211-47a0-979c-f8f6821e540a-secret-metrics-server-tls\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.381657 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.381597    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63191a58-2211-47a0-979c-f8f6821e540a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.381922 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.381630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/63191a58-2211-47a0-979c-f8f6821e540a-metrics-server-audit-profiles\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.381981 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.381944    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/63191a58-2211-47a0-979c-f8f6821e540a-audit-log\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.382608 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.382579    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63191a58-2211-47a0-979c-f8f6821e540a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.382736 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.382710    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/63191a58-2211-47a0-979c-f8f6821e540a-metrics-server-audit-profiles\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.384383 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.384355    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63191a58-2211-47a0-979c-f8f6821e540a-client-ca-bundle\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.384527 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.384509    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/63191a58-2211-47a0-979c-f8f6821e540a-secret-metrics-server-client-certs\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.384599 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.384543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/63191a58-2211-47a0-979c-f8f6821e540a-secret-metrics-server-tls\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.392843 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.392821    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqrgd\" (UniqueName: \"kubernetes.io/projected/63191a58-2211-47a0-979c-f8f6821e540a-kube-api-access-fqrgd\") pod \"metrics-server-db5bc955b-67wg2\" (UID: \"63191a58-2211-47a0-979c-f8f6821e540a\") " pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:53.547280 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:53.547193    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:33:54.168931 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.168901    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc" event={"ID":"a6454579-1b08-4ca5-bd1a-94284b0d0ecb","Type":"ContainerStarted","Data":"0c3df195232c2038a8d565f6f911dc91c762ae5840dbe7f4c6cc6e66c93c8b6b"}
Jul 16 10:33:54.186445 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.186370    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-db5bc955b-67wg2"]
Jul 16 10:33:54.513513 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.513481    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 10:33:54.517559 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.517532    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.520880 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.520788    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 16 10:33:54.520880 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.520821    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 16 10:33:54.520880 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.520834    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 16 10:33:54.521192 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.521097    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 16 10:33:54.521192 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.521132    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 16 10:33:54.521314 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.521280    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 16 10:33:54.521314 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.521278    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 16 10:33:54.521422 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.521329    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 16 10:33:54.521422 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.521352    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 16 10:33:54.521536 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.521481    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 16 10:33:54.521536 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.521509    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 16 10:33:54.521663 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.521481    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2gfdp\""
Jul 16 10:33:54.521663 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.521606    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5b4fjmqad7gar\""
Jul 16 10:33:54.521663 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.521635    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 16 10:33:54.524644 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.524613    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 16 10:33:54.528055 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.528033    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 16 10:33:54.539993 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.539961    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 10:33:54.592078 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592078 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592335 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592108    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592335 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592134    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592335 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592179    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592335 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592204    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592335 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592242    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-web-config\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592335 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592267    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5eb6cdd7-6ad7-4135-9123-274593bcf745-config-out\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592335 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592292    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592732 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592732 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592370    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592732 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592394    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592732 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592416    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592732 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592510    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592732 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592558    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-config\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592732 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592590    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rghj\" (UniqueName: \"kubernetes.io/projected/5eb6cdd7-6ad7-4135-9123-274593bcf745-kube-api-access-7rghj\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592732 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5eb6cdd7-6ad7-4135-9123-274593bcf745-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592732 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592685    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592732 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.592732 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.592718    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.693649 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.693584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.693649 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.693643    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-config\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.693926 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.693673    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rghj\" (UniqueName: \"kubernetes.io/projected/5eb6cdd7-6ad7-4135-9123-274593bcf745-kube-api-access-7rghj\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.693926 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.693701    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5eb6cdd7-6ad7-4135-9123-274593bcf745-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.693926 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.693717    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.693926 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.693735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.693926 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.693811    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.693926 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.693857    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.693926 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.693891    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.693926 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.693930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.694276 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.693957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.694276 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.693990    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.694276 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.694044    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.694276 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.694077    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-web-config\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.694276 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.694109    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5eb6cdd7-6ad7-4135-9123-274593bcf745-config-out\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.694276 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.694154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.694276 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.694190    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.694276 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.694217    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.694276 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.694245    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.694276 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.694270    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.695318 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.694793    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.695318 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.694833    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.695318 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.694866    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.695530 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.695345    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.696287 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.696259    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.696788 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.696766    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5eb6cdd7-6ad7-4135-9123-274593bcf745-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.697553 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.697226    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.697638 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.697557    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-web-config\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.697709 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.697685    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.698383 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.698356    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5eb6cdd7-6ad7-4135-9123-274593bcf745-config-out\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.698607 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.698587    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.698715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.698676    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.698839 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.698818    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.699280 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.699260    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-config\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.699672 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.699638    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.699963 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.699942    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.700211 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.700191    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.700551 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.700531    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.701215 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.701193    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.704893 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.704870    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rghj\" (UniqueName: \"kubernetes.io/projected/5eb6cdd7-6ad7-4135-9123-274593bcf745-kube-api-access-7rghj\") pod \"prometheus-k8s-0\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:54.828864 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:54.828771    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:33:55.041439 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:55.041102    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 10:33:55.056777 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:33:55.056721    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb6cdd7_6ad7_4135_9123_274593bcf745.slice/crio-8f4ddc3fb97ff2222a585efa786ac538779eb1040be2dc7a5dc2a555d394dbba WatchSource:0}: Error finding container 8f4ddc3fb97ff2222a585efa786ac538779eb1040be2dc7a5dc2a555d394dbba: Status 404 returned error can't find the container with id 8f4ddc3fb97ff2222a585efa786ac538779eb1040be2dc7a5dc2a555d394dbba
Jul 16 10:33:55.176300 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:55.176272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc" event={"ID":"a6454579-1b08-4ca5-bd1a-94284b0d0ecb","Type":"ContainerStarted","Data":"2d9ca1ac825e0a09173d8c70660edfc36f1d01005f5ecbb433bd5900bc7d20bf"}
Jul 16 10:33:55.176407 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:55.176365    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc" event={"ID":"a6454579-1b08-4ca5-bd1a-94284b0d0ecb","Type":"ContainerStarted","Data":"32d13b739c3329b25051bbaf497dcba2aaa2632ceff330258ccf92a144150f98"}
Jul 16 10:33:55.176464 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:55.176409    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc" event={"ID":"a6454579-1b08-4ca5-bd1a-94284b0d0ecb","Type":"ContainerStarted","Data":"fd3158508492a82a2bce2a83f573a42cbc6ac83f6d8312fc7bedc3a67a24a1fc"}
Jul 16 10:33:55.177591 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:55.177564    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerStarted","Data":"8f4ddc3fb97ff2222a585efa786ac538779eb1040be2dc7a5dc2a555d394dbba"}
Jul 16 10:33:55.178855 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:55.178833    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-db5bc955b-67wg2" event={"ID":"63191a58-2211-47a0-979c-f8f6821e540a","Type":"ContainerStarted","Data":"21eac4ddbc1795fda87b8aa4986219efc05788604fd3c14f8ac2e443cb5fd26a"}
Jul 16 10:33:56.187553 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:56.187508    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc" event={"ID":"a6454579-1b08-4ca5-bd1a-94284b0d0ecb","Type":"ContainerStarted","Data":"4df925a992ffb095c253922fabdf2bdc745599e5dffd18968376f0193f76c069"}
Jul 16 10:33:56.188035 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:56.187608    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc" event={"ID":"a6454579-1b08-4ca5-bd1a-94284b0d0ecb","Type":"ContainerStarted","Data":"6be5793b676059be8b37742672860256e2b630fe8613a59da4deefa30430f49d"}
Jul 16 10:33:56.188035 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:56.187652    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:33:56.189292 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:56.189262    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-db5bc955b-67wg2" event={"ID":"63191a58-2211-47a0-979c-f8f6821e540a","Type":"ContainerStarted","Data":"e6522711a3ffe224442c43f048dbce8072b19658d3efcf808dbd62873a741aec"}
Jul 16 10:33:56.224906 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:56.224826    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc" podStartSLOduration=2.077389116 podStartE2EDuration="5.224806774s" totalImagesPullingTime="3.147417658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:56.223003706 +0000 UTC m=+90.960038978" watchObservedRunningTime="2026-07-16 10:33:56.224806774 +0000 UTC m=+90.961842049"
Jul 16 10:33:56.250244 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:56.250190    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-db5bc955b-67wg2" podStartSLOduration=1.631414353 podStartE2EDuration="3.250173796s" totalImagesPullingTime="1.618759443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:56.249011203 +0000 UTC m=+90.986046474" watchObservedRunningTime="2026-07-16 10:33:56.250173796 +0000 UTC m=+90.987209066"
Jul 16 10:33:57.194842 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:57.194808    2576 generic.go:356] "Generic (PLEG): container finished" podID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerID="4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad" exitCode=0
Jul 16 10:33:57.195223 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:33:57.194890    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerDied","Data":"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"}
Jul 16 10:34:01.217039 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:01.217002    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerStarted","Data":"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"}
Jul 16 10:34:01.217477 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:01.217071    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerStarted","Data":"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"}
Jul 16 10:34:01.217477 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:01.217100    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerStarted","Data":"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"}
Jul 16 10:34:01.217477 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:01.217135    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerStarted","Data":"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"}
Jul 16 10:34:01.217477 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:01.217164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerStarted","Data":"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"}
Jul 16 10:34:01.217477 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:01.217189    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerStarted","Data":"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"}
Jul 16 10:34:01.255234 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:01.255137    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.5346382050000003 podStartE2EDuration="7.255115003s" totalImagesPullingTime="4.713001054s" totalInitContainerRuntime="7.475744ms" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:34:01.252917395 +0000 UTC m=+95.989952667" watchObservedRunningTime="2026-07-16 10:34:01.255115003 +0000 UTC m=+95.992150276"
Jul 16 10:34:02.201614 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:02.201585    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-c9b88f75f-8l7tc"
Jul 16 10:34:03.117931 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:03.117904    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-658996cf97-hss89"
Jul 16 10:34:04.829225 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:04.829187    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:34:13.115255 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:13.115220    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pxkgn"
Jul 16 10:34:13.547474 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:13.547398    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:34:13.547474 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:13.547452    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:34:33.553411 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:33.553379    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:34:33.557157 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:33.557135    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-db5bc955b-67wg2"
Jul 16 10:34:54.829516 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:54.829464    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:34:54.848336 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:54.848307    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:34:55.448560 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:34:55.448535    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.030412 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.030378    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 10:35:13.031029 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.030763    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="kube-rbac-proxy-thanos" containerID="cri-o://66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574" gracePeriod=600
Jul 16 10:35:13.031029 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.030783    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="prometheus" containerID="cri-o://a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823" gracePeriod=600
Jul 16 10:35:13.031029 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.030795    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="thanos-sidecar" containerID="cri-o://9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6" gracePeriod=600
Jul 16 10:35:13.031029 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.030821    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="config-reloader" containerID="cri-o://026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074" gracePeriod=600
Jul 16 10:35:13.031029 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.030778    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="kube-rbac-proxy-web" containerID="cri-o://77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9" gracePeriod=600
Jul 16 10:35:13.031266 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.031025    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="kube-rbac-proxy" containerID="cri-o://24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32" gracePeriod=600
Jul 16 10:35:13.284705 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.284640    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.383781 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.383729    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5eb6cdd7-6ad7-4135-9123-274593bcf745-tls-assets\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.383781 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.383785    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-metrics-client-ca\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384018 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.383805    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-2\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384018 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.383826    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-kube-rbac-proxy\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384018 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.383848    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5eb6cdd7-6ad7-4135-9123-274593bcf745-config-out\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384018 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.383875    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-1\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384224 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384199    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 10:35:13.384279 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384207    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-web-config\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384279 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384242    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 10:35:13.384386 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384278    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-db\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384386 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384320    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384386 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384360    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rghj\" (UniqueName: \"kubernetes.io/projected/5eb6cdd7-6ad7-4135-9123-274593bcf745-kube-api-access-7rghj\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384531 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384396    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-metrics-client-certs\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384531 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384484    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-kubelet-serving-ca-bundle\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384531 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384518    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384685 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384540    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 10:35:13.384685 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384551    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-config\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384685 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384603    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-serving-certs-ca-bundle\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384685 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384635    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-trusted-ca-bundle\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384685 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384668    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-grpc-tls\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384962 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384706    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-tls\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384962 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384732    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-thanos-prometheus-http-client-file\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.384962 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.384781    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-0\") pod \"5eb6cdd7-6ad7-4135-9123-274593bcf745\" (UID: \"5eb6cdd7-6ad7-4135-9123-274593bcf745\") "
Jul 16 10:35:13.385104 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.385003    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.385104 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.385018    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-metrics-client-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.385104 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.385033    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.385821 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.385682    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 10:35:13.386457 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.386434    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 10:35:13.386705 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.386622    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-config" (OuterVolumeSpecName: "config") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 10:35:13.386876 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.386847    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 10:35:13.387147 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.387098    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 10:35:13.387265 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.387243    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb6cdd7-6ad7-4135-9123-274593bcf745-config-out" (OuterVolumeSpecName: "config-out") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 10:35:13.387583 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.387465    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb6cdd7-6ad7-4135-9123-274593bcf745-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 10:35:13.387583 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.387544    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 10:35:13.387840 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.387807    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 10:35:13.388498 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.388471    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 10:35:13.389042 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.389016    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 10:35:13.389117 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.389038    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 10:35:13.389238 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.389215    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 10:35:13.389361 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.389342    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 10:35:13.389976 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.389959    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 10:35:13.390370 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.390346    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb6cdd7-6ad7-4135-9123-274593bcf745-kube-api-access-7rghj" (OuterVolumeSpecName: "kube-api-access-7rghj") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "kube-api-access-7rghj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 10:35:13.399255 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.399234    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-web-config" (OuterVolumeSpecName: "web-config") pod "5eb6cdd7-6ad7-4135-9123-274593bcf745" (UID: "5eb6cdd7-6ad7-4135-9123-274593bcf745"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 10:35:13.485887 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.485851    2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-kube-rbac-proxy\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.485887 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.485881    2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5eb6cdd7-6ad7-4135-9123-274593bcf745-config-out\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.485887 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.485893    2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-web-config\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.485905    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-db\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.485920    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.485933    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rghj\" (UniqueName: \"kubernetes.io/projected/5eb6cdd7-6ad7-4135-9123-274593bcf745-kube-api-access-7rghj\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.485943    2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-metrics-client-certs\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.485955    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.485968    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.485980    2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-config\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.485991    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.486005    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.486018    2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-grpc-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.486030    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.486042    2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5eb6cdd7-6ad7-4135-9123-274593bcf745-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.486054    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5eb6cdd7-6ad7-4135-9123-274593bcf745-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.486135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.486066    2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5eb6cdd7-6ad7-4135-9123-274593bcf745-tls-assets\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:35:13.499052 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499021    2576 generic.go:356] "Generic (PLEG): container finished" podID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerID="66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574" exitCode=0
Jul 16 10:35:13.499052 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499046    2576 generic.go:356] "Generic (PLEG): container finished" podID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerID="24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32" exitCode=0
Jul 16 10:35:13.499052 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499053    2576 generic.go:356] "Generic (PLEG): container finished" podID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerID="77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9" exitCode=0
Jul 16 10:35:13.499052 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499061    2576 generic.go:356] "Generic (PLEG): container finished" podID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerID="9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6" exitCode=0
Jul 16 10:35:13.499346 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499067    2576 generic.go:356] "Generic (PLEG): container finished" podID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerID="026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074" exitCode=0
Jul 16 10:35:13.499346 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499074    2576 generic.go:356] "Generic (PLEG): container finished" podID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerID="a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823" exitCode=0
Jul 16 10:35:13.499346 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499114    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.499346 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499105    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerDied","Data":"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"}
Jul 16 10:35:13.499346 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499252    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerDied","Data":"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"}
Jul 16 10:35:13.499346 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499280    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerDied","Data":"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"}
Jul 16 10:35:13.499346 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499306    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerDied","Data":"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"}
Jul 16 10:35:13.499346 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499328    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerDied","Data":"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"}
Jul 16 10:35:13.499346 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499351    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerDied","Data":"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"}
Jul 16 10:35:13.499735 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5eb6cdd7-6ad7-4135-9123-274593bcf745","Type":"ContainerDied","Data":"8f4ddc3fb97ff2222a585efa786ac538779eb1040be2dc7a5dc2a555d394dbba"}
Jul 16 10:35:13.499735 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.499329    2576 scope.go:117] "RemoveContainer" containerID="66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"
Jul 16 10:35:13.507273 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.507256    2576 scope.go:117] "RemoveContainer" containerID="24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"
Jul 16 10:35:13.514430 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.514406    2576 scope.go:117] "RemoveContainer" containerID="77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"
Jul 16 10:35:13.520733 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.520717    2576 scope.go:117] "RemoveContainer" containerID="9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"
Jul 16 10:35:13.528872 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.528588    2576 scope.go:117] "RemoveContainer" containerID="026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"
Jul 16 10:35:13.529644 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.529323    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 10:35:13.533387 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.533369    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 10:35:13.540597 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.540578    2576 scope.go:117] "RemoveContainer" containerID="a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"
Jul 16 10:35:13.547667 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.547650    2576 scope.go:117] "RemoveContainer" containerID="4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"
Jul 16 10:35:13.554537 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.554521    2576 scope.go:117] "RemoveContainer" containerID="66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"
Jul 16 10:35:13.554847 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:35:13.554820    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574\": container with ID starting with 66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574 not found: ID does not exist" containerID="66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"
Jul 16 10:35:13.554920 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.554859    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"} err="failed to get container status \"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574\": rpc error: code = NotFound desc = could not find container \"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574\": container with ID starting with 66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574 not found: ID does not exist"
Jul 16 10:35:13.554920 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.554902    2576 scope.go:117] "RemoveContainer" containerID="24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"
Jul 16 10:35:13.555155 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:35:13.555137    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32\": container with ID starting with 24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32 not found: ID does not exist" containerID="24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"
Jul 16 10:35:13.555214 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.555172    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"} err="failed to get container status \"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32\": rpc error: code = NotFound desc = could not find container \"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32\": container with ID starting with 24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32 not found: ID does not exist"
Jul 16 10:35:13.555214 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.555196    2576 scope.go:117] "RemoveContainer" containerID="77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"
Jul 16 10:35:13.555479 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:35:13.555462    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9\": container with ID starting with 77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9 not found: ID does not exist" containerID="77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"
Jul 16 10:35:13.555537 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.555483    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"} err="failed to get container status \"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9\": rpc error: code = NotFound desc = could not find container \"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9\": container with ID starting with 77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9 not found: ID does not exist"
Jul 16 10:35:13.555537 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.555497    2576 scope.go:117] "RemoveContainer" containerID="9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"
Jul 16 10:35:13.555767 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:35:13.555729    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6\": container with ID starting with 9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6 not found: ID does not exist" containerID="9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"
Jul 16 10:35:13.555767 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.555762    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"} err="failed to get container status \"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6\": rpc error: code = NotFound desc = could not find container \"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6\": container with ID starting with 9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6 not found: ID does not exist"
Jul 16 10:35:13.555917 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.555777    2576 scope.go:117] "RemoveContainer" containerID="026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"
Jul 16 10:35:13.555990 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:35:13.555975    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074\": container with ID starting with 026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074 not found: ID does not exist" containerID="026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"
Jul 16 10:35:13.556035 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.555998    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"} err="failed to get container status \"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074\": rpc error: code = NotFound desc = could not find container \"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074\": container with ID starting with 026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074 not found: ID does not exist"
Jul 16 10:35:13.556035 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.556012    2576 scope.go:117] "RemoveContainer" containerID="a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"
Jul 16 10:35:13.556285 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:35:13.556267    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823\": container with ID starting with a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823 not found: ID does not exist" containerID="a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"
Jul 16 10:35:13.556340 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.556289    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"} err="failed to get container status \"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823\": rpc error: code = NotFound desc = could not find container \"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823\": container with ID starting with a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823 not found: ID does not exist"
Jul 16 10:35:13.556340 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.556304    2576 scope.go:117] "RemoveContainer" containerID="4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"
Jul 16 10:35:13.556555 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:35:13.556537    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad\": container with ID starting with 4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad not found: ID does not exist" containerID="4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"
Jul 16 10:35:13.556612 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.556559    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"} err="failed to get container status \"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad\": rpc error: code = NotFound desc = could not find container \"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad\": container with ID starting with 4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad not found: ID does not exist"
Jul 16 10:35:13.556612 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.556572    2576 scope.go:117] "RemoveContainer" containerID="66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"
Jul 16 10:35:13.556825 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.556802    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"} err="failed to get container status \"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574\": rpc error: code = NotFound desc = could not find container \"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574\": container with ID starting with 66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574 not found: ID does not exist"
Jul 16 10:35:13.556891 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.556826    2576 scope.go:117] "RemoveContainer" containerID="24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"
Jul 16 10:35:13.557029 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.557011    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"} err="failed to get container status \"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32\": rpc error: code = NotFound desc = could not find container \"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32\": container with ID starting with 24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32 not found: ID does not exist"
Jul 16 10:35:13.557029 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.557027    2576 scope.go:117] "RemoveContainer" containerID="77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"
Jul 16 10:35:13.557271 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.557249    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"} err="failed to get container status \"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9\": rpc error: code = NotFound desc = could not find container \"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9\": container with ID starting with 77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9 not found: ID does not exist"
Jul 16 10:35:13.557331 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.557272    2576 scope.go:117] "RemoveContainer" containerID="9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"
Jul 16 10:35:13.557497 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.557480    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"} err="failed to get container status \"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6\": rpc error: code = NotFound desc = could not find container \"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6\": container with ID starting with 9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6 not found: ID does not exist"
Jul 16 10:35:13.557564 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.557497    2576 scope.go:117] "RemoveContainer" containerID="026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"
Jul 16 10:35:13.557729 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.557709    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"} err="failed to get container status \"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074\": rpc error: code = NotFound desc = could not find container \"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074\": container with ID starting with 026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074 not found: ID does not exist"
Jul 16 10:35:13.557792 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.557730    2576 scope.go:117] "RemoveContainer" containerID="a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"
Jul 16 10:35:13.557944 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.557926    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"} err="failed to get container status \"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823\": rpc error: code = NotFound desc = could not find container \"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823\": container with ID starting with a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823 not found: ID does not exist"
Jul 16 10:35:13.557991 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.557944    2576 scope.go:117] "RemoveContainer" containerID="4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"
Jul 16 10:35:13.558150 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.558135    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"} err="failed to get container status \"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad\": rpc error: code = NotFound desc = could not find container \"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad\": container with ID starting with 4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad not found: ID does not exist"
Jul 16 10:35:13.558200 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.558149    2576 scope.go:117] "RemoveContainer" containerID="66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"
Jul 16 10:35:13.558380 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.558362    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"} err="failed to get container status \"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574\": rpc error: code = NotFound desc = could not find container \"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574\": container with ID starting with 66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574 not found: ID does not exist"
Jul 16 10:35:13.558425 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.558380    2576 scope.go:117] "RemoveContainer" containerID="24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"
Jul 16 10:35:13.558592 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.558574    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"} err="failed to get container status \"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32\": rpc error: code = NotFound desc = could not find container \"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32\": container with ID starting with 24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32 not found: ID does not exist"
Jul 16 10:35:13.558634 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.558592    2576 scope.go:117] "RemoveContainer" containerID="77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"
Jul 16 10:35:13.558853 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.558836    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"} err="failed to get container status \"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9\": rpc error: code = NotFound desc = could not find container \"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9\": container with ID starting with 77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9 not found: ID does not exist"
Jul 16 10:35:13.558901 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.558853    2576 scope.go:117] "RemoveContainer" containerID="9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"
Jul 16 10:35:13.559069 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.559052    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"} err="failed to get container status \"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6\": rpc error: code = NotFound desc = could not find container \"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6\": container with ID starting with 9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6 not found: ID does not exist"
Jul 16 10:35:13.559069 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.559068    2576 scope.go:117] "RemoveContainer" containerID="026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"
Jul 16 10:35:13.559275 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.559257    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"} err="failed to get container status \"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074\": rpc error: code = NotFound desc = could not find container \"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074\": container with ID starting with 026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074 not found: ID does not exist"
Jul 16 10:35:13.559317 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.559276    2576 scope.go:117] "RemoveContainer" containerID="a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"
Jul 16 10:35:13.559478 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.559462    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"} err="failed to get container status \"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823\": rpc error: code = NotFound desc = could not find container \"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823\": container with ID starting with a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823 not found: ID does not exist"
Jul 16 10:35:13.559519 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.559478    2576 scope.go:117] "RemoveContainer" containerID="4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"
Jul 16 10:35:13.559655 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.559637    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"} err="failed to get container status \"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad\": rpc error: code = NotFound desc = could not find container \"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad\": container with ID starting with 4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad not found: ID does not exist"
Jul 16 10:35:13.559655 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.559654    2576 scope.go:117] "RemoveContainer" containerID="66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"
Jul 16 10:35:13.559869 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.559854    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"} err="failed to get container status \"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574\": rpc error: code = NotFound desc = could not find container \"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574\": container with ID starting with 66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574 not found: ID does not exist"
Jul 16 10:35:13.559869 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.559869    2576 scope.go:117] "RemoveContainer" containerID="24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"
Jul 16 10:35:13.560067 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.560052    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"} err="failed to get container status \"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32\": rpc error: code = NotFound desc = could not find container \"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32\": container with ID starting with 24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32 not found: ID does not exist"
Jul 16 10:35:13.560109 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.560068    2576 scope.go:117] "RemoveContainer" containerID="77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"
Jul 16 10:35:13.560272 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.560255    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"} err="failed to get container status \"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9\": rpc error: code = NotFound desc = could not find container \"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9\": container with ID starting with 77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9 not found: ID does not exist"
Jul 16 10:35:13.560316 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.560272    2576 scope.go:117] "RemoveContainer" containerID="9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"
Jul 16 10:35:13.560473 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.560452    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"} err="failed to get container status \"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6\": rpc error: code = NotFound desc = could not find container \"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6\": container with ID starting with 9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6 not found: ID does not exist"
Jul 16 10:35:13.560516 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.560473    2576 scope.go:117] "RemoveContainer" containerID="026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"
Jul 16 10:35:13.560675 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.560659    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"} err="failed to get container status \"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074\": rpc error: code = NotFound desc = could not find container \"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074\": container with ID starting with 026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074 not found: ID does not exist"
Jul 16 10:35:13.560675 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.560675    2576 scope.go:117] "RemoveContainer" containerID="a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"
Jul 16 10:35:13.560870 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.560854    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"} err="failed to get container status \"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823\": rpc error: code = NotFound desc = could not find container \"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823\": container with ID starting with a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823 not found: ID does not exist"
Jul 16 10:35:13.560922 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.560871    2576 scope.go:117] "RemoveContainer" containerID="4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"
Jul 16 10:35:13.561081 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.561064    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"} err="failed to get container status \"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad\": rpc error: code = NotFound desc = could not find container \"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad\": container with ID starting with 4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad not found: ID does not exist"
Jul 16 10:35:13.561132 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.561081    2576 scope.go:117] "RemoveContainer" containerID="66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"
Jul 16 10:35:13.561268 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.561252    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"} err="failed to get container status \"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574\": rpc error: code = NotFound desc = could not find container \"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574\": container with ID starting with 66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574 not found: ID does not exist"
Jul 16 10:35:13.561268 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.561267    2576 scope.go:117] "RemoveContainer" containerID="24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"
Jul 16 10:35:13.561437 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.561421    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"} err="failed to get container status \"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32\": rpc error: code = NotFound desc = could not find container \"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32\": container with ID starting with 24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32 not found: ID does not exist"
Jul 16 10:35:13.561437 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.561435    2576 scope.go:117] "RemoveContainer" containerID="77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"
Jul 16 10:35:13.561591 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.561577    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"} err="failed to get container status \"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9\": rpc error: code = NotFound desc = could not find container \"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9\": container with ID starting with 77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9 not found: ID does not exist"
Jul 16 10:35:13.561642 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.561592    2576 scope.go:117] "RemoveContainer" containerID="9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"
Jul 16 10:35:13.561805 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.561787    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"} err="failed to get container status \"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6\": rpc error: code = NotFound desc = could not find container \"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6\": container with ID starting with 9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6 not found: ID does not exist"
Jul 16 10:35:13.561862 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.561806    2576 scope.go:117] "RemoveContainer" containerID="026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"
Jul 16 10:35:13.561973 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.561956    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"} err="failed to get container status \"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074\": rpc error: code = NotFound desc = could not find container \"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074\": container with ID starting with 026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074 not found: ID does not exist"
Jul 16 10:35:13.561973 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.561970    2576 scope.go:117] "RemoveContainer" containerID="a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"
Jul 16 10:35:13.562295 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.562263    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"} err="failed to get container status \"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823\": rpc error: code = NotFound desc = could not find container \"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823\": container with ID starting with a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823 not found: ID does not exist"
Jul 16 10:35:13.562415 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.562320    2576 scope.go:117] "RemoveContainer" containerID="4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"
Jul 16 10:35:13.562651 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.562631    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"} err="failed to get container status \"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad\": rpc error: code = NotFound desc = could not find container \"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad\": container with ID starting with 4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad not found: ID does not exist"
Jul 16 10:35:13.562735 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.562653    2576 scope.go:117] "RemoveContainer" containerID="66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"
Jul 16 10:35:13.562987 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.562969    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574"} err="failed to get container status \"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574\": rpc error: code = NotFound desc = could not find container \"66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574\": container with ID starting with 66d25b3ee0f042e575e3a26a955bd6dec4d3b94e200f5882ccbccd47aaa4e574 not found: ID does not exist"
Jul 16 10:35:13.563071 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.562998    2576 scope.go:117] "RemoveContainer" containerID="24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"
Jul 16 10:35:13.563375 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.563356    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32"} err="failed to get container status \"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32\": rpc error: code = NotFound desc = could not find container \"24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32\": container with ID starting with 24cb6f17f2f6b09a8b7b6f70b313e7c23c31a4f2b4093fdb809f1fb66da63c32 not found: ID does not exist"
Jul 16 10:35:13.563375 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.563375    2576 scope.go:117] "RemoveContainer" containerID="77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"
Jul 16 10:35:13.563643 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.563625    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9"} err="failed to get container status \"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9\": rpc error: code = NotFound desc = could not find container \"77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9\": container with ID starting with 77f05e79fa3d3d31a9aec4fd83725b10bb2f21061ea4226070e26c0d8126c2a9 not found: ID does not exist"
Jul 16 10:35:13.563704 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.563644    2576 scope.go:117] "RemoveContainer" containerID="9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"
Jul 16 10:35:13.563906 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.563885    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6"} err="failed to get container status \"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6\": rpc error: code = NotFound desc = could not find container \"9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6\": container with ID starting with 9eaef64855caf778e7e90b5e6414db378f444531d45bcfad6979363700a60ee6 not found: ID does not exist"
Jul 16 10:35:13.563968 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.563908    2576 scope.go:117] "RemoveContainer" containerID="026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"
Jul 16 10:35:13.564105 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564087    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 10:35:13.564164 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564101    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074"} err="failed to get container status \"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074\": rpc error: code = NotFound desc = could not find container \"026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074\": container with ID starting with 026dca66df936aaa7e5431202ac5a058f2dec1b5fd4cb1f4faff704332647074 not found: ID does not exist"
Jul 16 10:35:13.564164 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564132    2576 scope.go:117] "RemoveContainer" containerID="a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"
Jul 16 10:35:13.564352 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564327    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823"} err="failed to get container status \"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823\": rpc error: code = NotFound desc = could not find container \"a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823\": container with ID starting with a60e33d0b6e0058f2a2fd9b992c0814e11b5f9a8d56c5d7e108e7a7ed4777823 not found: ID does not exist"
Jul 16 10:35:13.564421 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564353    2576 scope.go:117] "RemoveContainer" containerID="4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"
Jul 16 10:35:13.564421 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564383    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="config-reloader"
Jul 16 10:35:13.564421 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564395    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="config-reloader"
Jul 16 10:35:13.564421 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564408    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="prometheus"
Jul 16 10:35:13.564421 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564414    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="prometheus"
Jul 16 10:35:13.564421 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564419    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="kube-rbac-proxy"
Jul 16 10:35:13.564421 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564425    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="kube-rbac-proxy"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564432    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="thanos-sidecar"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564437    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="thanos-sidecar"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564460    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="kube-rbac-proxy-thanos"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564466    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="kube-rbac-proxy-thanos"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564474    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="kube-rbac-proxy-web"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564479    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="kube-rbac-proxy-web"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564514    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="init-config-reloader"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564520    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="init-config-reloader"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564573    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="thanos-sidecar"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564565    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad"} err="failed to get container status \"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad\": rpc error: code = NotFound desc = could not find container \"4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad\": container with ID starting with 4ccdb4775fc0671c58fba6e4876c2092de65f866c7ed94c797e3e641fc60b0ad not found: ID does not exist"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564580    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="prometheus"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564597    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="config-reloader"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564606    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="kube-rbac-proxy-web"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564615    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="kube-rbac-proxy-thanos"
Jul 16 10:35:13.564715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.564628    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" containerName="kube-rbac-proxy"
Jul 16 10:35:13.569973 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.569954    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.572632 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.572609    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 16 10:35:13.572723 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.572675    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 16 10:35:13.572723 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.572705    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 16 10:35:13.572723 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.572713    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 16 10:35:13.572970 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.572756    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 16 10:35:13.572970 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.572676    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 16 10:35:13.573031 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.573000    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5b4fjmqad7gar\""
Jul 16 10:35:13.573220 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.573207    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 16 10:35:13.573256 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.573235    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 16 10:35:13.573559 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.573547    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 16 10:35:13.573717 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.573695    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2gfdp\""
Jul 16 10:35:13.573875 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.573699    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 16 10:35:13.573875 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.573768    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 16 10:35:13.573986 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.573972    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 16 10:35:13.577193 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.577073    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 16 10:35:13.580227 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.580203    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 16 10:35:13.586610 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586589    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f076ad8d-df9e-4335-bf64-17174228fd5e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.586706 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586615    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.586706 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586640    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.586706 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586659    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.586706 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586682    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.586706 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.586953 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586721    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.586953 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586792    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f076ad8d-df9e-4335-bf64-17174228fd5e-config-out\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.586953 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586851    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.586953 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586912    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.586953 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586934    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.586953 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg4s6\" (UniqueName: \"kubernetes.io/projected/f076ad8d-df9e-4335-bf64-17174228fd5e-kube-api-access-kg4s6\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.587223 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586966    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.587223 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.586998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-config\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.587223 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.587018    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.587223 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.587036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.587223 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.587061    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.587223 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.587131    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-web-config\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.587223 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.587168    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.587223 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.587193    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.590039 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.590019    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 10:35:13.687543 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.687509    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.687778 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.687550    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-web-config\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.687778 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.687578    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.687778 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.687603    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.687778 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.687630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f076ad8d-df9e-4335-bf64-17174228fd5e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.687778 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.687661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.687778 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.687690    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.687778 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.687716    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.687778 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.687738    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.687984    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688037    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688067    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f076ad8d-df9e-4335-bf64-17174228fd5e-config-out\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688158    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688206    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg4s6\" (UniqueName: \"kubernetes.io/projected/f076ad8d-df9e-4335-bf64-17174228fd5e-kube-api-access-kg4s6\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688232    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688256    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-config\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688302    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688359    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688502    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.688604    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689832 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.689133    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.689832 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.689235    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.690283 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.690254    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.691166 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.691121    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f076ad8d-df9e-4335-bf64-17174228fd5e-config-out\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.691252 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.691175    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.691388 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.691363    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-web-config\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.691458 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.691430    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.692266 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.692222    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f076ad8d-df9e-4335-bf64-17174228fd5e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.692670 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.692648    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f076ad8d-df9e-4335-bf64-17174228fd5e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.692773 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.692738    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.693027 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.692999    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.693120 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.693069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.693176 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.693154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.693300 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.693284    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-config\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.693864 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.693848    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f076ad8d-df9e-4335-bf64-17174228fd5e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.698807 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.698791    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg4s6\" (UniqueName: \"kubernetes.io/projected/f076ad8d-df9e-4335-bf64-17174228fd5e-kube-api-access-kg4s6\") pod \"prometheus-k8s-0\" (UID: \"f076ad8d-df9e-4335-bf64-17174228fd5e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:13.800355 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.800283    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5eb6cdd7-6ad7-4135-9123-274593bcf745" path="/var/lib/kubelet/pods/5eb6cdd7-6ad7-4135-9123-274593bcf745/volumes"
Jul 16 10:35:13.882279 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:13.882247    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:35:14.027385 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:14.027352    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 10:35:14.030614 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:35:14.030589    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf076ad8d_df9e_4335_bf64_17174228fd5e.slice/crio-649670eeef7f94f53d31660c9befe17ff955cccd69139aa99c96bd5d12702f9d WatchSource:0}: Error finding container 649670eeef7f94f53d31660c9befe17ff955cccd69139aa99c96bd5d12702f9d: Status 404 returned error can't find the container with id 649670eeef7f94f53d31660c9befe17ff955cccd69139aa99c96bd5d12702f9d
Jul 16 10:35:14.505019 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:14.504987    2576 generic.go:356] "Generic (PLEG): container finished" podID="f076ad8d-df9e-4335-bf64-17174228fd5e" containerID="10b37fc452d9576f116ffcbabb1865b53161d597f47581fec157e667674d702a" exitCode=0
Jul 16 10:35:14.505171 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:14.505032    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f076ad8d-df9e-4335-bf64-17174228fd5e","Type":"ContainerDied","Data":"10b37fc452d9576f116ffcbabb1865b53161d597f47581fec157e667674d702a"}
Jul 16 10:35:14.505171 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:14.505083    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f076ad8d-df9e-4335-bf64-17174228fd5e","Type":"ContainerStarted","Data":"649670eeef7f94f53d31660c9befe17ff955cccd69139aa99c96bd5d12702f9d"}
Jul 16 10:35:15.511828 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:15.511789    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f076ad8d-df9e-4335-bf64-17174228fd5e","Type":"ContainerStarted","Data":"1035650fd75c47304a13541992001788c3c9b8030c0bd20f56a185ba7d5b1f4d"}
Jul 16 10:35:15.512193 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:15.511884    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f076ad8d-df9e-4335-bf64-17174228fd5e","Type":"ContainerStarted","Data":"961fc2a786a931878c29f725988a78eb404315183561368b32fd1fb9f34d55b3"}
Jul 16 10:35:15.512193 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:15.511911    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f076ad8d-df9e-4335-bf64-17174228fd5e","Type":"ContainerStarted","Data":"3333300d90c6dd5b0d133c5617ebd47056dff1aabb235a17deefbaa969931c38"}
Jul 16 10:35:15.512193 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:15.511935    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f076ad8d-df9e-4335-bf64-17174228fd5e","Type":"ContainerStarted","Data":"98105eca51828c3ddb430c0621ea7bb2dfbab5b391d8eb30211282790ec7f9a0"}
Jul 16 10:35:15.512193 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:15.511959    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f076ad8d-df9e-4335-bf64-17174228fd5e","Type":"ContainerStarted","Data":"9768650f0f1f7d91c164d104e969172308bdc1cdaed3782773ce2047db8a5fa2"}
Jul 16 10:35:15.512193 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:15.511982    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f076ad8d-df9e-4335-bf64-17174228fd5e","Type":"ContainerStarted","Data":"66087b36a45c1355bb684cc64ac9031c9627e3ced8bd46f33d214c31e873a46b"}
Jul 16 10:35:18.882777 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:35:18.882678    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:36:13.883117 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:36:13.883083    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:36:13.898555 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:36:13.898533    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:36:13.934633 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:36:13.934581    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=60.915387148 podStartE2EDuration="1m0.934566052s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.178904ms" isStatefulPod=false podCreationTimestamp="2026-07-16 10:35:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:35:15.550832921 +0000 UTC m=+170.287868195" watchObservedRunningTime="2026-07-16 10:36:13.934566052 +0000 UTC m=+228.671601323"
Jul 16 10:36:14.753300 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:36:14.753271    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 10:37:23.278704 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.278668    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"]
Jul 16 10:37:23.281964 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.281948    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.290475 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.290451    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 16 10:37:23.290738 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.290716    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 16 10:37:23.290849 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.290758    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 16 10:37:23.290849 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.290764    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 16 10:37:23.290936 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.290857    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-j9zwf\""
Jul 16 10:37:23.327455 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.327424    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"]
Jul 16 10:37:23.355369 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.355322    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l2wj\" (UniqueName: \"kubernetes.io/projected/2bb0e822-72d3-4622-8f58-aed1b5ee4add-kube-api-access-2l2wj\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.355508 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.355389    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bb0e822-72d3-4622-8f58-aed1b5ee4add-tmp\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.355508 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.355435    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bb0e822-72d3-4622-8f58-aed1b5ee4add-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.355587 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.355503    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2bb0e822-72d3-4622-8f58-aed1b5ee4add-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.355587 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.355545    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2bb0e822-72d3-4622-8f58-aed1b5ee4add-manifests\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.399586 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.399552    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"]
Jul 16 10:37:23.402802 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.402783    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.407252 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.407235    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 16 10:37:23.407363 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.407275    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 16 10:37:23.407363 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.407325    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 16 10:37:23.407363 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.407275    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 16 10:37:23.407529 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.407275    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bzbdm\""
Jul 16 10:37:23.407647 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.407632    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 16 10:37:23.417909 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.417890    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"]
Jul 16 10:37:23.456453 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.456424    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcfg8\" (UniqueName: \"kubernetes.io/projected/aca3864e-af4d-455e-8ee1-dbfa3fc433d0-kube-api-access-mcfg8\") pod \"lws-controller-manager-85564789c6-8xbvg\" (UID: \"aca3864e-af4d-455e-8ee1-dbfa3fc433d0\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.456599 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.456470    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bb0e822-72d3-4622-8f58-aed1b5ee4add-tmp\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.456599 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.456485    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aca3864e-af4d-455e-8ee1-dbfa3fc433d0-cert\") pod \"lws-controller-manager-85564789c6-8xbvg\" (UID: \"aca3864e-af4d-455e-8ee1-dbfa3fc433d0\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.456599 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.456543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bb0e822-72d3-4622-8f58-aed1b5ee4add-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.456599 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.456567    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aca3864e-af4d-455e-8ee1-dbfa3fc433d0-metrics-cert\") pod \"lws-controller-manager-85564789c6-8xbvg\" (UID: \"aca3864e-af4d-455e-8ee1-dbfa3fc433d0\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.456810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.456605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2bb0e822-72d3-4622-8f58-aed1b5ee4add-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.456810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.456624    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aca3864e-af4d-455e-8ee1-dbfa3fc433d0-manager-config\") pod \"lws-controller-manager-85564789c6-8xbvg\" (UID: \"aca3864e-af4d-455e-8ee1-dbfa3fc433d0\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.456810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.456644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2bb0e822-72d3-4622-8f58-aed1b5ee4add-manifests\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.456810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.456668    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l2wj\" (UniqueName: \"kubernetes.io/projected/2bb0e822-72d3-4622-8f58-aed1b5ee4add-kube-api-access-2l2wj\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.457032 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.457014    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2bb0e822-72d3-4622-8f58-aed1b5ee4add-manifests\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.458670 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.458647    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bb0e822-72d3-4622-8f58-aed1b5ee4add-tmp\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.458978 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.458961    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bb0e822-72d3-4622-8f58-aed1b5ee4add-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.459026 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.459013    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2bb0e822-72d3-4622-8f58-aed1b5ee4add-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.475964 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.475938    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l2wj\" (UniqueName: \"kubernetes.io/projected/2bb0e822-72d3-4622-8f58-aed1b5ee4add-kube-api-access-2l2wj\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-wff4j\" (UID: \"2bb0e822-72d3-4622-8f58-aed1b5ee4add\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.557725 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.557639    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aca3864e-af4d-455e-8ee1-dbfa3fc433d0-metrics-cert\") pod \"lws-controller-manager-85564789c6-8xbvg\" (UID: \"aca3864e-af4d-455e-8ee1-dbfa3fc433d0\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.557725 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.557698    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aca3864e-af4d-455e-8ee1-dbfa3fc433d0-manager-config\") pod \"lws-controller-manager-85564789c6-8xbvg\" (UID: \"aca3864e-af4d-455e-8ee1-dbfa3fc433d0\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.557963 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.557737    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcfg8\" (UniqueName: \"kubernetes.io/projected/aca3864e-af4d-455e-8ee1-dbfa3fc433d0-kube-api-access-mcfg8\") pod \"lws-controller-manager-85564789c6-8xbvg\" (UID: \"aca3864e-af4d-455e-8ee1-dbfa3fc433d0\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.557963 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.557785    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aca3864e-af4d-455e-8ee1-dbfa3fc433d0-cert\") pod \"lws-controller-manager-85564789c6-8xbvg\" (UID: \"aca3864e-af4d-455e-8ee1-dbfa3fc433d0\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.558331 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.558312    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aca3864e-af4d-455e-8ee1-dbfa3fc433d0-manager-config\") pod \"lws-controller-manager-85564789c6-8xbvg\" (UID: \"aca3864e-af4d-455e-8ee1-dbfa3fc433d0\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.560078 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.560056    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aca3864e-af4d-455e-8ee1-dbfa3fc433d0-metrics-cert\") pod \"lws-controller-manager-85564789c6-8xbvg\" (UID: \"aca3864e-af4d-455e-8ee1-dbfa3fc433d0\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.560211 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.560191    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aca3864e-af4d-455e-8ee1-dbfa3fc433d0-cert\") pod \"lws-controller-manager-85564789c6-8xbvg\" (UID: \"aca3864e-af4d-455e-8ee1-dbfa3fc433d0\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.576197 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.576173    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcfg8\" (UniqueName: \"kubernetes.io/projected/aca3864e-af4d-455e-8ee1-dbfa3fc433d0-kube-api-access-mcfg8\") pod \"lws-controller-manager-85564789c6-8xbvg\" (UID: \"aca3864e-af4d-455e-8ee1-dbfa3fc433d0\") " pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.592190 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.592170    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:23.712080 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.712047    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:23.738309 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.738166    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"]
Jul 16 10:37:23.741057 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:37:23.741029    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb0e822_72d3_4622_8f58_aed1b5ee4add.slice/crio-3b6ab643819b9dce2fd5f5c4bf68abc32702e80dd949d4aae72a15b77305cffb WatchSource:0}: Error finding container 3b6ab643819b9dce2fd5f5c4bf68abc32702e80dd949d4aae72a15b77305cffb: Status 404 returned error can't find the container with id 3b6ab643819b9dce2fd5f5c4bf68abc32702e80dd949d4aae72a15b77305cffb
Jul 16 10:37:23.874964 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.874938    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"]
Jul 16 10:37:23.877073 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:37:23.877049    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaca3864e_af4d_455e_8ee1_dbfa3fc433d0.slice/crio-10ac55b0abe709958a1c96221b16e65df5cbd790976c7728276d242c2e8d4b9f WatchSource:0}: Error finding container 10ac55b0abe709958a1c96221b16e65df5cbd790976c7728276d242c2e8d4b9f: Status 404 returned error can't find the container with id 10ac55b0abe709958a1c96221b16e65df5cbd790976c7728276d242c2e8d4b9f
Jul 16 10:37:23.991138 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.991103    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg" event={"ID":"aca3864e-af4d-455e-8ee1-dbfa3fc433d0","Type":"ContainerStarted","Data":"10ac55b0abe709958a1c96221b16e65df5cbd790976c7728276d242c2e8d4b9f"}
Jul 16 10:37:23.992298 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:23.992279    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j" event={"ID":"2bb0e822-72d3-4622-8f58-aed1b5ee4add","Type":"ContainerStarted","Data":"3b6ab643819b9dce2fd5f5c4bf68abc32702e80dd949d4aae72a15b77305cffb"}
Jul 16 10:37:27.780536 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:27.780509    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:37:27.782939 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:27.780576    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:37:27.786911 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:27.786894    2576 kubelet.go:1648] "Image garbage collection succeeded"
Jul 16 10:37:28.017797 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:28.017754    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j" event={"ID":"2bb0e822-72d3-4622-8f58-aed1b5ee4add","Type":"ContainerStarted","Data":"b856fff81841f941d38af1717bd0755891cd32d43432f35a93f26b166b3e3495"}
Jul 16 10:37:28.019260 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:28.019236    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg" event={"ID":"aca3864e-af4d-455e-8ee1-dbfa3fc433d0","Type":"ContainerStarted","Data":"8f58370df30ab7d0542c51c9ff0b866c6cba4c60e0cbca16c4c6270c11c2feae"}
Jul 16 10:37:28.019395 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:28.019378    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:28.072459 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:28.072364    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg" podStartSLOduration=1.114667035 podStartE2EDuration="5.072348453s" totalImagesPullingTime="3.957681418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:37:28.071247744 +0000 UTC m=+302.808283006" watchObservedRunningTime="2026-07-16 10:37:28.072348453 +0000 UTC m=+302.809383723"
Jul 16 10:37:29.025613 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:29.025581    2576 generic.go:356] "Generic (PLEG): container finished" podID="2bb0e822-72d3-4622-8f58-aed1b5ee4add" containerID="b856fff81841f941d38af1717bd0755891cd32d43432f35a93f26b166b3e3495" exitCode=0
Jul 16 10:37:29.025994 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:29.025674    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j" event={"ID":"2bb0e822-72d3-4622-8f58-aed1b5ee4add","Type":"ContainerDied","Data":"b856fff81841f941d38af1717bd0755891cd32d43432f35a93f26b166b3e3495"}
Jul 16 10:37:29.026191 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:29.026176    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 10:37:31.036058 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:31.036012    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j" event={"ID":"2bb0e822-72d3-4622-8f58-aed1b5ee4add","Type":"ContainerStarted","Data":"e942020daf7a9bb1102817010f5410f9814dd5384503598bd26c9d1ed7078e5b"}
Jul 16 10:37:31.036658 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:31.036178    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:31.061756 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:31.061693    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j" podStartSLOduration=2.004741487 podStartE2EDuration="8.06167411s" totalImagesPullingTime="5.039913256s" totalInitContainerRuntime="1.017019367s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:37:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:37:31.059694796 +0000 UTC m=+305.796730081" watchObservedRunningTime="2026-07-16 10:37:31.06167411 +0000 UTC m=+305.798709381"
Jul 16 10:37:39.028451 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:39.028420    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-85564789c6-8xbvg"
Jul 16 10:37:42.042467 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:42.042442    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-wff4j"
Jul 16 10:37:45.563067 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.563030    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"]
Jul 16 10:37:45.575263 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.575232    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"
Jul 16 10:37:45.579007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.578981    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 16 10:37:45.579150 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.579029    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 16 10:37:45.579150 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.579046    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 16 10:37:45.579150 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.579074    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 16 10:37:45.582120 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.581977    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vtpv6\""
Jul 16 10:37:45.615171 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.615140    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"]
Jul 16 10:37:45.756642 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.756612    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4db6a21-4d71-4e75-b7f7-c3e1979c1d88-tmp\") pod \"kube-auth-proxy-58946d4b5c-pb828\" (UID: \"f4db6a21-4d71-4e75-b7f7-c3e1979c1d88\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"
Jul 16 10:37:45.756845 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.756655    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljdrl\" (UniqueName: \"kubernetes.io/projected/f4db6a21-4d71-4e75-b7f7-c3e1979c1d88-kube-api-access-ljdrl\") pod \"kube-auth-proxy-58946d4b5c-pb828\" (UID: \"f4db6a21-4d71-4e75-b7f7-c3e1979c1d88\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"
Jul 16 10:37:45.756845 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.756723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4db6a21-4d71-4e75-b7f7-c3e1979c1d88-tls-certs\") pod \"kube-auth-proxy-58946d4b5c-pb828\" (UID: \"f4db6a21-4d71-4e75-b7f7-c3e1979c1d88\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"
Jul 16 10:37:45.858207 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.858174    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4db6a21-4d71-4e75-b7f7-c3e1979c1d88-tmp\") pod \"kube-auth-proxy-58946d4b5c-pb828\" (UID: \"f4db6a21-4d71-4e75-b7f7-c3e1979c1d88\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"
Jul 16 10:37:45.858390 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.858216    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljdrl\" (UniqueName: \"kubernetes.io/projected/f4db6a21-4d71-4e75-b7f7-c3e1979c1d88-kube-api-access-ljdrl\") pod \"kube-auth-proxy-58946d4b5c-pb828\" (UID: \"f4db6a21-4d71-4e75-b7f7-c3e1979c1d88\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"
Jul 16 10:37:45.858390 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.858246    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4db6a21-4d71-4e75-b7f7-c3e1979c1d88-tls-certs\") pod \"kube-auth-proxy-58946d4b5c-pb828\" (UID: \"f4db6a21-4d71-4e75-b7f7-c3e1979c1d88\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"
Jul 16 10:37:45.860385 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.860356    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4db6a21-4d71-4e75-b7f7-c3e1979c1d88-tmp\") pod \"kube-auth-proxy-58946d4b5c-pb828\" (UID: \"f4db6a21-4d71-4e75-b7f7-c3e1979c1d88\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"
Jul 16 10:37:45.860678 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.860654    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4db6a21-4d71-4e75-b7f7-c3e1979c1d88-tls-certs\") pod \"kube-auth-proxy-58946d4b5c-pb828\" (UID: \"f4db6a21-4d71-4e75-b7f7-c3e1979c1d88\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"
Jul 16 10:37:45.873997 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.873971    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljdrl\" (UniqueName: \"kubernetes.io/projected/f4db6a21-4d71-4e75-b7f7-c3e1979c1d88-kube-api-access-ljdrl\") pod \"kube-auth-proxy-58946d4b5c-pb828\" (UID: \"f4db6a21-4d71-4e75-b7f7-c3e1979c1d88\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"
Jul 16 10:37:45.884836 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:45.884817    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"
Jul 16 10:37:46.075567 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:46.075541    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58946d4b5c-pb828"]
Jul 16 10:37:46.077122 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:37:46.077097    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4db6a21_4d71_4e75_b7f7_c3e1979c1d88.slice/crio-d889630ca3bc86b8ad9b101f46666b0b80f4e8a8766279f4d2f663c56ff3b4b6 WatchSource:0}: Error finding container d889630ca3bc86b8ad9b101f46666b0b80f4e8a8766279f4d2f663c56ff3b4b6: Status 404 returned error can't find the container with id d889630ca3bc86b8ad9b101f46666b0b80f4e8a8766279f4d2f663c56ff3b4b6
Jul 16 10:37:46.102223 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:46.102189    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828" event={"ID":"f4db6a21-4d71-4e75-b7f7-c3e1979c1d88","Type":"ContainerStarted","Data":"d889630ca3bc86b8ad9b101f46666b0b80f4e8a8766279f4d2f663c56ff3b4b6"}
Jul 16 10:37:49.116292 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:49.116260    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828" event={"ID":"f4db6a21-4d71-4e75-b7f7-c3e1979c1d88","Type":"ContainerStarted","Data":"b070f4d676bd8beee2af70911da6a3b3b00e760742570f18aa314658c98efe39"}
Jul 16 10:37:49.142375 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:37:49.142330    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-58946d4b5c-pb828" podStartSLOduration=1.85967417 podStartE2EDuration="4.142313823s" totalImagesPullingTime="2.282639653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:37:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:37:49.141720674 +0000 UTC m=+323.878755946" watchObservedRunningTime="2026-07-16 10:37:49.142313823 +0000 UTC m=+323.879349134"
Jul 16 10:39:46.732530 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:46.732495    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"]
Jul 16 10:39:46.735660 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:46.735645    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"
Jul 16 10:39:46.739878 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:46.739858    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 10:39:46.740637 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:46.740609    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 10:39:46.740794 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:46.740695    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-g4tqf\""
Jul 16 10:39:46.759654 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:46.759626    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"]
Jul 16 10:39:46.785579 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:46.785544    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73747b0d-6488-4c6f-a2a7-941c3dece66d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hgf9v\" (UID: \"73747b0d-6488-4c6f-a2a7-941c3dece66d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"
Jul 16 10:39:46.785717 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:46.785676    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6kb2\" (UniqueName: \"kubernetes.io/projected/73747b0d-6488-4c6f-a2a7-941c3dece66d-kube-api-access-b6kb2\") pod \"kuadrant-operator-controller-manager-86878c769f-hgf9v\" (UID: \"73747b0d-6488-4c6f-a2a7-941c3dece66d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"
Jul 16 10:39:46.886781 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:46.886716    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73747b0d-6488-4c6f-a2a7-941c3dece66d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hgf9v\" (UID: \"73747b0d-6488-4c6f-a2a7-941c3dece66d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"
Jul 16 10:39:46.886965 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:46.886833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6kb2\" (UniqueName: \"kubernetes.io/projected/73747b0d-6488-4c6f-a2a7-941c3dece66d-kube-api-access-b6kb2\") pod \"kuadrant-operator-controller-manager-86878c769f-hgf9v\" (UID: \"73747b0d-6488-4c6f-a2a7-941c3dece66d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"
Jul 16 10:39:46.887168 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:46.887147    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73747b0d-6488-4c6f-a2a7-941c3dece66d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hgf9v\" (UID: \"73747b0d-6488-4c6f-a2a7-941c3dece66d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"
Jul 16 10:39:46.906832 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:46.906802    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6kb2\" (UniqueName: \"kubernetes.io/projected/73747b0d-6488-4c6f-a2a7-941c3dece66d-kube-api-access-b6kb2\") pod \"kuadrant-operator-controller-manager-86878c769f-hgf9v\" (UID: \"73747b0d-6488-4c6f-a2a7-941c3dece66d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"
Jul 16 10:39:47.046192 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:47.046079    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"
Jul 16 10:39:47.240988 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:47.240964    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"]
Jul 16 10:39:47.243443 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:39:47.243417    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73747b0d_6488_4c6f_a2a7_941c3dece66d.slice/crio-af057570cc616bb4e5fadcd70f20d916f6988dd66ffbffddd19ef791d729d503 WatchSource:0}: Error finding container af057570cc616bb4e5fadcd70f20d916f6988dd66ffbffddd19ef791d729d503: Status 404 returned error can't find the container with id af057570cc616bb4e5fadcd70f20d916f6988dd66ffbffddd19ef791d729d503
Jul 16 10:39:47.610362 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:47.610329    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v" event={"ID":"73747b0d-6488-4c6f-a2a7-941c3dece66d","Type":"ContainerStarted","Data":"af057570cc616bb4e5fadcd70f20d916f6988dd66ffbffddd19ef791d729d503"}
Jul 16 10:39:52.637601 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:52.637564    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v" event={"ID":"73747b0d-6488-4c6f-a2a7-941c3dece66d","Type":"ContainerStarted","Data":"31646e036dee29217c2b718952e9b9980287d933e457df08b524b97194544864"}
Jul 16 10:39:52.637994 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:52.637758    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"
Jul 16 10:39:52.678477 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:39:52.678422    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v" podStartSLOduration=2.116862263 podStartE2EDuration="6.67840415s" totalImagesPullingTime="4.561541887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:39:52.678033723 +0000 UTC m=+447.415068995" watchObservedRunningTime="2026-07-16 10:39:52.67840415 +0000 UTC m=+447.415439474"
Jul 16 10:40:03.644169 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:03.644138    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"
Jul 16 10:40:27.958766 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:27.958715    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-qqs2p"]
Jul 16 10:40:27.961179 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:27.961157    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qqs2p"
Jul 16 10:40:27.967268 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:27.967246    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fd6bn\""
Jul 16 10:40:27.976803 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:27.976776    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-qqs2p"]
Jul 16 10:40:28.051099 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.051060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfkvb\" (UniqueName: \"kubernetes.io/projected/00f627e7-1733-4293-a4a6-d25bc065307a-kube-api-access-mfkvb\") pod \"authorino-58cd955675-qqs2p\" (UID: \"00f627e7-1733-4293-a4a6-d25bc065307a\") " pod="kuadrant-system/authorino-58cd955675-qqs2p"
Jul 16 10:40:28.152259 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.152203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfkvb\" (UniqueName: \"kubernetes.io/projected/00f627e7-1733-4293-a4a6-d25bc065307a-kube-api-access-mfkvb\") pod \"authorino-58cd955675-qqs2p\" (UID: \"00f627e7-1733-4293-a4a6-d25bc065307a\") " pod="kuadrant-system/authorino-58cd955675-qqs2p"
Jul 16 10:40:28.168358 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.168332    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfkvb\" (UniqueName: \"kubernetes.io/projected/00f627e7-1733-4293-a4a6-d25bc065307a-kube-api-access-mfkvb\") pod \"authorino-58cd955675-qqs2p\" (UID: \"00f627e7-1733-4293-a4a6-d25bc065307a\") " pod="kuadrant-system/authorino-58cd955675-qqs2p"
Jul 16 10:40:28.270020 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.269928    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qqs2p"
Jul 16 10:40:28.333334 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.333299    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-jvkhc"]
Jul 16 10:40:28.336716 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.336691    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jvkhc"
Jul 16 10:40:28.352283 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.352214    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jvkhc"]
Jul 16 10:40:28.405768 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.405712    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-qqs2p"]
Jul 16 10:40:28.409046 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:40:28.409018    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00f627e7_1733_4293_a4a6_d25bc065307a.slice/crio-3fa44818f4b681b9a356b165f44969e1603fa47353562376436507cb53ef4876 WatchSource:0}: Error finding container 3fa44818f4b681b9a356b165f44969e1603fa47353562376436507cb53ef4876: Status 404 returned error can't find the container with id 3fa44818f4b681b9a356b165f44969e1603fa47353562376436507cb53ef4876
Jul 16 10:40:28.455365 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.455333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njbjw\" (UniqueName: \"kubernetes.io/projected/973d4554-af5e-407b-a217-ac353be3ccae-kube-api-access-njbjw\") pod \"authorino-75d556bd77-jvkhc\" (UID: \"973d4554-af5e-407b-a217-ac353be3ccae\") " pod="kuadrant-system/authorino-75d556bd77-jvkhc"
Jul 16 10:40:28.556802 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.556701    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njbjw\" (UniqueName: \"kubernetes.io/projected/973d4554-af5e-407b-a217-ac353be3ccae-kube-api-access-njbjw\") pod \"authorino-75d556bd77-jvkhc\" (UID: \"973d4554-af5e-407b-a217-ac353be3ccae\") " pod="kuadrant-system/authorino-75d556bd77-jvkhc"
Jul 16 10:40:28.566677 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.566650    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njbjw\" (UniqueName: \"kubernetes.io/projected/973d4554-af5e-407b-a217-ac353be3ccae-kube-api-access-njbjw\") pod \"authorino-75d556bd77-jvkhc\" (UID: \"973d4554-af5e-407b-a217-ac353be3ccae\") " pod="kuadrant-system/authorino-75d556bd77-jvkhc"
Jul 16 10:40:28.646305 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.646259    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jvkhc"
Jul 16 10:40:28.775648 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.775617    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jvkhc"]
Jul 16 10:40:28.780248 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:40:28.780220    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod973d4554_af5e_407b_a217_ac353be3ccae.slice/crio-fc54fe46467c1a66a21e66e6b1285fab4165cd733f2f32145e044b6ed0d5b409 WatchSource:0}: Error finding container fc54fe46467c1a66a21e66e6b1285fab4165cd733f2f32145e044b6ed0d5b409: Status 404 returned error can't find the container with id fc54fe46467c1a66a21e66e6b1285fab4165cd733f2f32145e044b6ed0d5b409
Jul 16 10:40:28.790752 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.790714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qqs2p" event={"ID":"00f627e7-1733-4293-a4a6-d25bc065307a","Type":"ContainerStarted","Data":"3fa44818f4b681b9a356b165f44969e1603fa47353562376436507cb53ef4876"}
Jul 16 10:40:28.791667 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:28.791649    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jvkhc" event={"ID":"973d4554-af5e-407b-a217-ac353be3ccae","Type":"ContainerStarted","Data":"fc54fe46467c1a66a21e66e6b1285fab4165cd733f2f32145e044b6ed0d5b409"}
Jul 16 10:40:32.818578 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:32.818518    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qqs2p" event={"ID":"00f627e7-1733-4293-a4a6-d25bc065307a","Type":"ContainerStarted","Data":"5b17f7b858aad56cf81decda0052c94b23ab33f75f7ea956684e4cfb067d44e2"}
Jul 16 10:40:32.820046 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:32.820025    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jvkhc" event={"ID":"973d4554-af5e-407b-a217-ac353be3ccae","Type":"ContainerStarted","Data":"b49e4113f28cde7d496a743c792ab6323a3c3df55da3ea09d3d93a4d031cdef8"}
Jul 16 10:40:32.843443 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:32.843397    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-qqs2p" podStartSLOduration=2.5029834920000003 podStartE2EDuration="5.8433792s" totalImagesPullingTime="3.340395708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:40:32.84119241 +0000 UTC m=+487.578227681" watchObservedRunningTime="2026-07-16 10:40:32.8433792 +0000 UTC m=+487.580414470"
Jul 16 10:40:32.862089 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:32.862026    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-jvkhc" podStartSLOduration=1.8808999979999999 podStartE2EDuration="4.862007869s" totalImagesPullingTime="2.981107871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:40:32.860357357 +0000 UTC m=+487.597392627" watchObservedRunningTime="2026-07-16 10:40:32.862007869 +0000 UTC m=+487.599043142"
Jul 16 10:40:32.899271 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:32.899238    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-qqs2p"]
Jul 16 10:40:34.829043 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:34.829003    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-qqs2p" podUID="00f627e7-1733-4293-a4a6-d25bc065307a" containerName="authorino" containerID="cri-o://5b17f7b858aad56cf81decda0052c94b23ab33f75f7ea956684e4cfb067d44e2" gracePeriod=30
Jul 16 10:40:35.074997 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:35.074972    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qqs2p"
Jul 16 10:40:35.220721 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:35.220676    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfkvb\" (UniqueName: \"kubernetes.io/projected/00f627e7-1733-4293-a4a6-d25bc065307a-kube-api-access-mfkvb\") pod \"00f627e7-1733-4293-a4a6-d25bc065307a\" (UID: \"00f627e7-1733-4293-a4a6-d25bc065307a\") "
Jul 16 10:40:35.222792 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:35.222762    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00f627e7-1733-4293-a4a6-d25bc065307a-kube-api-access-mfkvb" (OuterVolumeSpecName: "kube-api-access-mfkvb") pod "00f627e7-1733-4293-a4a6-d25bc065307a" (UID: "00f627e7-1733-4293-a4a6-d25bc065307a"). InnerVolumeSpecName "kube-api-access-mfkvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 10:40:35.322030 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:35.321987    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfkvb\" (UniqueName: \"kubernetes.io/projected/00f627e7-1733-4293-a4a6-d25bc065307a-kube-api-access-mfkvb\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:40:35.834364 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:35.834332    2576 generic.go:356] "Generic (PLEG): container finished" podID="00f627e7-1733-4293-a4a6-d25bc065307a" containerID="5b17f7b858aad56cf81decda0052c94b23ab33f75f7ea956684e4cfb067d44e2" exitCode=0
Jul 16 10:40:35.834730 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:35.834367    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qqs2p"
Jul 16 10:40:35.834730 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:35.834388    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qqs2p" event={"ID":"00f627e7-1733-4293-a4a6-d25bc065307a","Type":"ContainerDied","Data":"5b17f7b858aad56cf81decda0052c94b23ab33f75f7ea956684e4cfb067d44e2"}
Jul 16 10:40:35.834730 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:35.834435    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qqs2p" event={"ID":"00f627e7-1733-4293-a4a6-d25bc065307a","Type":"ContainerDied","Data":"3fa44818f4b681b9a356b165f44969e1603fa47353562376436507cb53ef4876"}
Jul 16 10:40:35.834730 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:35.834455    2576 scope.go:117] "RemoveContainer" containerID="5b17f7b858aad56cf81decda0052c94b23ab33f75f7ea956684e4cfb067d44e2"
Jul 16 10:40:35.842361 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:35.842343    2576 scope.go:117] "RemoveContainer" containerID="5b17f7b858aad56cf81decda0052c94b23ab33f75f7ea956684e4cfb067d44e2"
Jul 16 10:40:35.842637 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:40:35.842619    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b17f7b858aad56cf81decda0052c94b23ab33f75f7ea956684e4cfb067d44e2\": container with ID starting with 5b17f7b858aad56cf81decda0052c94b23ab33f75f7ea956684e4cfb067d44e2 not found: ID does not exist" containerID="5b17f7b858aad56cf81decda0052c94b23ab33f75f7ea956684e4cfb067d44e2"
Jul 16 10:40:35.842679 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:35.842645    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b17f7b858aad56cf81decda0052c94b23ab33f75f7ea956684e4cfb067d44e2"} err="failed to get container status \"5b17f7b858aad56cf81decda0052c94b23ab33f75f7ea956684e4cfb067d44e2\": rpc error: code = NotFound desc = could not find container \"5b17f7b858aad56cf81decda0052c94b23ab33f75f7ea956684e4cfb067d44e2\": container with ID starting with 5b17f7b858aad56cf81decda0052c94b23ab33f75f7ea956684e4cfb067d44e2 not found: ID does not exist"
Jul 16 10:40:35.856920 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:35.856895    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-qqs2p"]
Jul 16 10:40:35.858934 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:35.858913    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-qqs2p"]
Jul 16 10:40:37.800686 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:40:37.800651    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00f627e7-1733-4293-a4a6-d25bc065307a" path="/var/lib/kubelet/pods/00f627e7-1733-4293-a4a6-d25bc065307a/volumes"
Jul 16 10:41:01.221997 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.221960    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-798f678c77-wd46v"]
Jul 16 10:41:01.222602 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.222462    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f627e7-1733-4293-a4a6-d25bc065307a" containerName="authorino"
Jul 16 10:41:01.222602 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.222483    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f627e7-1733-4293-a4a6-d25bc065307a" containerName="authorino"
Jul 16 10:41:01.222602 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.222586    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="00f627e7-1733-4293-a4a6-d25bc065307a" containerName="authorino"
Jul 16 10:41:01.228526 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.228506    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-798f678c77-wd46v"
Jul 16 10:41:01.232121 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.232099    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 16 10:41:01.239696 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.239674    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-798f678c77-wd46v"]
Jul 16 10:41:01.347996 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.347962    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msbmd\" (UniqueName: \"kubernetes.io/projected/d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a-kube-api-access-msbmd\") pod \"authorino-798f678c77-wd46v\" (UID: \"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a\") " pod="kuadrant-system/authorino-798f678c77-wd46v"
Jul 16 10:41:01.348155 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.348007    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a-tls-cert\") pod \"authorino-798f678c77-wd46v\" (UID: \"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a\") " pod="kuadrant-system/authorino-798f678c77-wd46v"
Jul 16 10:41:01.448892 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.448860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msbmd\" (UniqueName: \"kubernetes.io/projected/d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a-kube-api-access-msbmd\") pod \"authorino-798f678c77-wd46v\" (UID: \"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a\") " pod="kuadrant-system/authorino-798f678c77-wd46v"
Jul 16 10:41:01.449064 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.448899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a-tls-cert\") pod \"authorino-798f678c77-wd46v\" (UID: \"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a\") " pod="kuadrant-system/authorino-798f678c77-wd46v"
Jul 16 10:41:01.451336 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.451305    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a-tls-cert\") pod \"authorino-798f678c77-wd46v\" (UID: \"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a\") " pod="kuadrant-system/authorino-798f678c77-wd46v"
Jul 16 10:41:01.459274 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.459248    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msbmd\" (UniqueName: \"kubernetes.io/projected/d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a-kube-api-access-msbmd\") pod \"authorino-798f678c77-wd46v\" (UID: \"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a\") " pod="kuadrant-system/authorino-798f678c77-wd46v"
Jul 16 10:41:01.538175 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.538081    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-798f678c77-wd46v"
Jul 16 10:41:01.676904 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.676871    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-798f678c77-wd46v"]
Jul 16 10:41:01.680421 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:41:01.680397    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1a79b79_61e6_4e4c_8e4c_17e45fa2bf3a.slice/crio-dec1b3f01701ef8e284585664fb79568e51e6e9ad34675e0cc98ed13adf659de WatchSource:0}: Error finding container dec1b3f01701ef8e284585664fb79568e51e6e9ad34675e0cc98ed13adf659de: Status 404 returned error can't find the container with id dec1b3f01701ef8e284585664fb79568e51e6e9ad34675e0cc98ed13adf659de
Jul 16 10:41:01.946236 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:01.946197    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-798f678c77-wd46v" event={"ID":"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a","Type":"ContainerStarted","Data":"dec1b3f01701ef8e284585664fb79568e51e6e9ad34675e0cc98ed13adf659de"}
Jul 16 10:41:02.951590 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:02.951556    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-798f678c77-wd46v" event={"ID":"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a","Type":"ContainerStarted","Data":"1231bace2b72414b4fd17487472abbc7778c53554c800c6f66d85430fce59038"}
Jul 16 10:41:02.974232 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:02.974181    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-798f678c77-wd46v" podStartSLOduration=1.673036267 podStartE2EDuration="1.974164742s" totalImagesPullingTime="301.128475ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:41:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:41:02.971986166 +0000 UTC m=+517.709021448" watchObservedRunningTime="2026-07-16 10:41:02.974164742 +0000 UTC m=+517.711200013"
Jul 16 10:41:03.016701 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.016669    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jvkhc"]
Jul 16 10:41:03.016873 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.016850    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-jvkhc" podUID="973d4554-af5e-407b-a217-ac353be3ccae" containerName="authorino" containerID="cri-o://b49e4113f28cde7d496a743c792ab6323a3c3df55da3ea09d3d93a4d031cdef8" gracePeriod=30
Jul 16 10:41:03.272826 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.272805    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jvkhc"
Jul 16 10:41:03.363397 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.363363    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njbjw\" (UniqueName: \"kubernetes.io/projected/973d4554-af5e-407b-a217-ac353be3ccae-kube-api-access-njbjw\") pod \"973d4554-af5e-407b-a217-ac353be3ccae\" (UID: \"973d4554-af5e-407b-a217-ac353be3ccae\") "
Jul 16 10:41:03.365412 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.365382    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/973d4554-af5e-407b-a217-ac353be3ccae-kube-api-access-njbjw" (OuterVolumeSpecName: "kube-api-access-njbjw") pod "973d4554-af5e-407b-a217-ac353be3ccae" (UID: "973d4554-af5e-407b-a217-ac353be3ccae"). InnerVolumeSpecName "kube-api-access-njbjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 10:41:03.464636 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.464606    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njbjw\" (UniqueName: \"kubernetes.io/projected/973d4554-af5e-407b-a217-ac353be3ccae-kube-api-access-njbjw\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:41:03.957221 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.957186    2576 generic.go:356] "Generic (PLEG): container finished" podID="973d4554-af5e-407b-a217-ac353be3ccae" containerID="b49e4113f28cde7d496a743c792ab6323a3c3df55da3ea09d3d93a4d031cdef8" exitCode=0
Jul 16 10:41:03.957618 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.957231    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jvkhc"
Jul 16 10:41:03.957618 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.957249    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jvkhc" event={"ID":"973d4554-af5e-407b-a217-ac353be3ccae","Type":"ContainerDied","Data":"b49e4113f28cde7d496a743c792ab6323a3c3df55da3ea09d3d93a4d031cdef8"}
Jul 16 10:41:03.957618 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.957297    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jvkhc" event={"ID":"973d4554-af5e-407b-a217-ac353be3ccae","Type":"ContainerDied","Data":"fc54fe46467c1a66a21e66e6b1285fab4165cd733f2f32145e044b6ed0d5b409"}
Jul 16 10:41:03.957618 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.957318    2576 scope.go:117] "RemoveContainer" containerID="b49e4113f28cde7d496a743c792ab6323a3c3df55da3ea09d3d93a4d031cdef8"
Jul 16 10:41:03.967531 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.967511    2576 scope.go:117] "RemoveContainer" containerID="b49e4113f28cde7d496a743c792ab6323a3c3df55da3ea09d3d93a4d031cdef8"
Jul 16 10:41:03.968013 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:41:03.967973    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b49e4113f28cde7d496a743c792ab6323a3c3df55da3ea09d3d93a4d031cdef8\": container with ID starting with b49e4113f28cde7d496a743c792ab6323a3c3df55da3ea09d3d93a4d031cdef8 not found: ID does not exist" containerID="b49e4113f28cde7d496a743c792ab6323a3c3df55da3ea09d3d93a4d031cdef8"
Jul 16 10:41:03.968124 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.968016    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b49e4113f28cde7d496a743c792ab6323a3c3df55da3ea09d3d93a4d031cdef8"} err="failed to get container status \"b49e4113f28cde7d496a743c792ab6323a3c3df55da3ea09d3d93a4d031cdef8\": rpc error: code = NotFound desc = could not find container \"b49e4113f28cde7d496a743c792ab6323a3c3df55da3ea09d3d93a4d031cdef8\": container with ID starting with b49e4113f28cde7d496a743c792ab6323a3c3df55da3ea09d3d93a4d031cdef8 not found: ID does not exist"
Jul 16 10:41:03.977907 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.977884    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jvkhc"]
Jul 16 10:41:03.980063 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:03.980044    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jvkhc"]
Jul 16 10:41:05.800801 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:41:05.800769    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="973d4554-af5e-407b-a217-ac353be3ccae" path="/var/lib/kubelet/pods/973d4554-af5e-407b-a217-ac353be3ccae/volumes"
Jul 16 10:42:04.929242 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:04.929203    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"]
Jul 16 10:42:04.929644 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:04.929544    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="973d4554-af5e-407b-a217-ac353be3ccae" containerName="authorino"
Jul 16 10:42:04.929644 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:04.929554    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="973d4554-af5e-407b-a217-ac353be3ccae" containerName="authorino"
Jul 16 10:42:04.929644 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:04.929616    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="973d4554-af5e-407b-a217-ac353be3ccae" containerName="authorino"
Jul 16 10:42:04.932776 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:04.932738    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:04.936616 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:04.936590    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 16 10:42:04.942878 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:04.942856    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 16 10:42:04.942994 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:04.942934    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 16 10:42:04.943082 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:04.943068    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6m4hr\""
Jul 16 10:42:04.947314 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:04.947293    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"]
Jul 16 10:42:05.006380 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.006344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.006380 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.006382    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.006604 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.006410    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.006604 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.006429    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.006604 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.006481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.006604 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.006542    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fl8q\" (UniqueName: \"kubernetes.io/projected/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-kube-api-access-4fl8q\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.107994 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.107963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.107994 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.107996    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.108209 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.108023    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.108209 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.108040    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.108209 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.108059    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.108209 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.108108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fl8q\" (UniqueName: \"kubernetes.io/projected/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-kube-api-access-4fl8q\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.108485 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.108455    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.108612 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.108520    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.108612 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.108587    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.110287 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.110265    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.110434 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.110414    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.117122 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.117073    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fl8q\" (UniqueName: \"kubernetes.io/projected/c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3-kube-api-access-4fl8q\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn\" (UID: \"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.243150 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.243064    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:05.381795 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:05.381767    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"]
Jul 16 10:42:05.384340 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:42:05.384318    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc601eca5_7ba4_4ab9_9d5f_d6e4f0f5dcc3.slice/crio-868ba2e61ca7018f05372b49e8d71f929e361d8c69b6626146714bdf17d7a87e WatchSource:0}: Error finding container 868ba2e61ca7018f05372b49e8d71f929e361d8c69b6626146714bdf17d7a87e: Status 404 returned error can't find the container with id 868ba2e61ca7018f05372b49e8d71f929e361d8c69b6626146714bdf17d7a87e
Jul 16 10:42:06.230346 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:06.230311    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn" event={"ID":"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3","Type":"ContainerStarted","Data":"868ba2e61ca7018f05372b49e8d71f929e361d8c69b6626146714bdf17d7a87e"}
Jul 16 10:42:07.236660 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:07.236620    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn" event={"ID":"c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3","Type":"ContainerStarted","Data":"39e761f35538ca5731027a4b43f1f438fa60042b35a22d94a699ad2ed0a7c0c3"}
Jul 16 10:42:07.262902 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:07.262856    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn" podStartSLOduration=2.11282128 podStartE2EDuration="3.262841054s" totalImagesPullingTime="1.150019774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:42:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:42:07.261186906 +0000 UTC m=+581.998222188" watchObservedRunningTime="2026-07-16 10:42:07.262841054 +0000 UTC m=+581.999876324"
Jul 16 10:42:15.243376 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:15.243275    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:15.243879 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:15.243427    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:15.261543 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:15.261509    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:15.288030 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:15.288002    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn"
Jul 16 10:42:27.809845 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:27.809721    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:42:27.815352 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:27.810350    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:42:31.747177 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.745254    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"]
Jul 16 10:42:31.750952 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.750929    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.754794 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.754775    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 16 10:42:31.778807 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.778782    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"]
Jul 16 10:42:31.849389 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.849358    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ba13897-88c8-48a9-a35a-d088eeeff396-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.849546 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.849414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m22zh\" (UniqueName: \"kubernetes.io/projected/5ba13897-88c8-48a9-a35a-d088eeeff396-kube-api-access-m22zh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.849546 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.849482    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ba13897-88c8-48a9-a35a-d088eeeff396-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.849546 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.849518    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ba13897-88c8-48a9-a35a-d088eeeff396-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.849546 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.849545    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ba13897-88c8-48a9-a35a-d088eeeff396-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.849682 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.849562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ba13897-88c8-48a9-a35a-d088eeeff396-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.950516 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.950478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ba13897-88c8-48a9-a35a-d088eeeff396-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.950719 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.950546    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m22zh\" (UniqueName: \"kubernetes.io/projected/5ba13897-88c8-48a9-a35a-d088eeeff396-kube-api-access-m22zh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.950719 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.950576    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ba13897-88c8-48a9-a35a-d088eeeff396-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.950719 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.950601    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ba13897-88c8-48a9-a35a-d088eeeff396-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.950719 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.950629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ba13897-88c8-48a9-a35a-d088eeeff396-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.950719 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.950652    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ba13897-88c8-48a9-a35a-d088eeeff396-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.951038 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.950999    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ba13897-88c8-48a9-a35a-d088eeeff396-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.951038 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.951022    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ba13897-88c8-48a9-a35a-d088eeeff396-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.951115 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.951057    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ba13897-88c8-48a9-a35a-d088eeeff396-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.952911 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.952893    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ba13897-88c8-48a9-a35a-d088eeeff396-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.953229 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.953212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ba13897-88c8-48a9-a35a-d088eeeff396-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:31.959562 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:31.959537    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m22zh\" (UniqueName: \"kubernetes.io/projected/5ba13897-88c8-48a9-a35a-d088eeeff396-kube-api-access-m22zh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n\" (UID: \"5ba13897-88c8-48a9-a35a-d088eeeff396\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:32.061065 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:32.060983    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:32.194819 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:32.194795    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"]
Jul 16 10:42:32.197305 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:42:32.197279    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba13897_88c8_48a9_a35a_d088eeeff396.slice/crio-d7986134bcd520118b7bb0842eff188e1f007f32e7e72a08cbb3cbb51462f558 WatchSource:0}: Error finding container d7986134bcd520118b7bb0842eff188e1f007f32e7e72a08cbb3cbb51462f558: Status 404 returned error can't find the container with id d7986134bcd520118b7bb0842eff188e1f007f32e7e72a08cbb3cbb51462f558
Jul 16 10:42:32.198783 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:32.198765    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 10:42:32.343644 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:32.343554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n" event={"ID":"5ba13897-88c8-48a9-a35a-d088eeeff396","Type":"ContainerStarted","Data":"d7986134bcd520118b7bb0842eff188e1f007f32e7e72a08cbb3cbb51462f558"}
Jul 16 10:42:33.348715 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:33.348679    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n" event={"ID":"5ba13897-88c8-48a9-a35a-d088eeeff396","Type":"ContainerStarted","Data":"12b9bd697e080d91e4822e6949acd8d847c2dd536893613a918c73e9d45bd0c6"}
Jul 16 10:42:33.373552 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:33.373484    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n" podStartSLOduration=2.195155178 podStartE2EDuration="2.373468221s" totalImagesPullingTime="178.313043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:42:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:42:33.372778749 +0000 UTC m=+608.109814012" watchObservedRunningTime="2026-07-16 10:42:33.373468221 +0000 UTC m=+608.110503492"
Jul 16 10:42:40.432336 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.432290    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"]
Jul 16 10:42:40.437359 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.437331    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.442821 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.442796    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 16 10:42:40.458613 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.458587    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"]
Jul 16 10:42:40.528039 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.527993    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58a948c0-efda-4aab-a1bd-151f990f11ef-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.528226 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.528042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58a948c0-efda-4aab-a1bd-151f990f11ef-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.528226 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.528143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kbnr\" (UniqueName: \"kubernetes.io/projected/58a948c0-efda-4aab-a1bd-151f990f11ef-kube-api-access-4kbnr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.528226 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.528181    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58a948c0-efda-4aab-a1bd-151f990f11ef-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.528226 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.528211    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58a948c0-efda-4aab-a1bd-151f990f11ef-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.528417 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.528280    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58a948c0-efda-4aab-a1bd-151f990f11ef-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.629078 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.629042    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58a948c0-efda-4aab-a1bd-151f990f11ef-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.629243 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.629083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58a948c0-efda-4aab-a1bd-151f990f11ef-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.629243 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.629117    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kbnr\" (UniqueName: \"kubernetes.io/projected/58a948c0-efda-4aab-a1bd-151f990f11ef-kube-api-access-4kbnr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.629243 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.629137    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58a948c0-efda-4aab-a1bd-151f990f11ef-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.629243 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.629166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58a948c0-efda-4aab-a1bd-151f990f11ef-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.629243 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.629228    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58a948c0-efda-4aab-a1bd-151f990f11ef-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.629567 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.629541    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58a948c0-efda-4aab-a1bd-151f990f11ef-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.629848 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.629816    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58a948c0-efda-4aab-a1bd-151f990f11ef-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.630004 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.629982    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58a948c0-efda-4aab-a1bd-151f990f11ef-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.631637 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.631614    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58a948c0-efda-4aab-a1bd-151f990f11ef-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.632028 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.632007    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58a948c0-efda-4aab-a1bd-151f990f11ef-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.643262 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.643233    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kbnr\" (UniqueName: \"kubernetes.io/projected/58a948c0-efda-4aab-a1bd-151f990f11ef-kube-api-access-4kbnr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h\" (UID: \"58a948c0-efda-4aab-a1bd-151f990f11ef\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.747241 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.747159    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:40.891637 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:40.891602    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"]
Jul 16 10:42:41.396274 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:41.396224    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h" event={"ID":"58a948c0-efda-4aab-a1bd-151f990f11ef","Type":"ContainerStarted","Data":"6cc289b1e73dbc4c2825e39ecd0b91d21c3597ed10fecfaa27a5a3155da387f8"}
Jul 16 10:42:41.396274 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:41.396279    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h" event={"ID":"58a948c0-efda-4aab-a1bd-151f990f11ef","Type":"ContainerStarted","Data":"4e695da6ef4231a695d5a446a082ccc493cbe3402e0b69f966167fa2ab2e9028"}
Jul 16 10:42:41.427347 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:41.427293    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h" podStartSLOduration=1.250683055 podStartE2EDuration="1.427278548s" totalImagesPullingTime="176.595493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:42:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:42:41.425437182 +0000 UTC m=+616.162472463" watchObservedRunningTime="2026-07-16 10:42:41.427278548 +0000 UTC m=+616.164313818"
Jul 16 10:42:42.061155 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:42.061120    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:42.061526 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:42.061181    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:42.073530 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:42.073501    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:42.411656 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:42.411631    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n"
Jul 16 10:42:50.747763 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:50.747702    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:50.748231 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:50.747782    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:50.759941 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:50.759915    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:42:51.452160 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:42:51.452131    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h"
Jul 16 10:44:02.241977 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.241939    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-798f678c77-wd46v"]
Jul 16 10:44:02.242487 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.242130    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-798f678c77-wd46v" podUID="d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a" containerName="authorino" containerID="cri-o://1231bace2b72414b4fd17487472abbc7778c53554c800c6f66d85430fce59038" gracePeriod=30
Jul 16 10:44:02.485810 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.485787    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-798f678c77-wd46v"
Jul 16 10:44:02.555648 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.555575    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msbmd\" (UniqueName: \"kubernetes.io/projected/d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a-kube-api-access-msbmd\") pod \"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a\" (UID: \"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a\") "
Jul 16 10:44:02.555648 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.555604    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a-tls-cert\") pod \"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a\" (UID: \"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a\") "
Jul 16 10:44:02.557699 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.557672    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a-kube-api-access-msbmd" (OuterVolumeSpecName: "kube-api-access-msbmd") pod "d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a" (UID: "d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a"). InnerVolumeSpecName "kube-api-access-msbmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 10:44:02.565044 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.565024    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a" (UID: "d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 10:44:02.656945 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.656913    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msbmd\" (UniqueName: \"kubernetes.io/projected/d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a-kube-api-access-msbmd\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:44:02.656945 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.656939    2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a-tls-cert\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:44:02.754882 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.754852    2576 generic.go:356] "Generic (PLEG): container finished" podID="d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a" containerID="1231bace2b72414b4fd17487472abbc7778c53554c800c6f66d85430fce59038" exitCode=0
Jul 16 10:44:02.755037 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.754898    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-798f678c77-wd46v"
Jul 16 10:44:02.755037 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.754933    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-798f678c77-wd46v" event={"ID":"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a","Type":"ContainerDied","Data":"1231bace2b72414b4fd17487472abbc7778c53554c800c6f66d85430fce59038"}
Jul 16 10:44:02.755037 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.754978    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-798f678c77-wd46v" event={"ID":"d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a","Type":"ContainerDied","Data":"dec1b3f01701ef8e284585664fb79568e51e6e9ad34675e0cc98ed13adf659de"}
Jul 16 10:44:02.755037 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.754995    2576 scope.go:117] "RemoveContainer" containerID="1231bace2b72414b4fd17487472abbc7778c53554c800c6f66d85430fce59038"
Jul 16 10:44:02.763494 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.763477    2576 scope.go:117] "RemoveContainer" containerID="1231bace2b72414b4fd17487472abbc7778c53554c800c6f66d85430fce59038"
Jul 16 10:44:02.763737 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:44:02.763717    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1231bace2b72414b4fd17487472abbc7778c53554c800c6f66d85430fce59038\": container with ID starting with 1231bace2b72414b4fd17487472abbc7778c53554c800c6f66d85430fce59038 not found: ID does not exist" containerID="1231bace2b72414b4fd17487472abbc7778c53554c800c6f66d85430fce59038"
Jul 16 10:44:02.763799 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.763760    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1231bace2b72414b4fd17487472abbc7778c53554c800c6f66d85430fce59038"} err="failed to get container status \"1231bace2b72414b4fd17487472abbc7778c53554c800c6f66d85430fce59038\": rpc error: code = NotFound desc = could not find container \"1231bace2b72414b4fd17487472abbc7778c53554c800c6f66d85430fce59038\": container with ID starting with 1231bace2b72414b4fd17487472abbc7778c53554c800c6f66d85430fce59038 not found: ID does not exist"
Jul 16 10:44:02.780787 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.780762    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-798f678c77-wd46v"]
Jul 16 10:44:02.789383 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:02.789366    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-798f678c77-wd46v"]
Jul 16 10:44:03.800007 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:44:03.799975    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a" path="/var/lib/kubelet/pods/d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a/volumes"
Jul 16 10:47:27.840896 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:47:27.840785    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:47:27.846884 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:47:27.843066    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:50:09.613198 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:09.613160    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"]
Jul 16 10:50:09.613732 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:09.613341    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v" podUID="73747b0d-6488-4c6f-a2a7-941c3dece66d" containerName="manager" containerID="cri-o://31646e036dee29217c2b718952e9b9980287d933e457df08b524b97194544864" gracePeriod=10
Jul 16 10:50:12.370419 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:12.370385    2576 generic.go:356] "Generic (PLEG): container finished" podID="73747b0d-6488-4c6f-a2a7-941c3dece66d" containerID="31646e036dee29217c2b718952e9b9980287d933e457df08b524b97194544864" exitCode=0
Jul 16 10:50:12.370895 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:12.370452    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v" event={"ID":"73747b0d-6488-4c6f-a2a7-941c3dece66d","Type":"ContainerDied","Data":"31646e036dee29217c2b718952e9b9980287d933e457df08b524b97194544864"}
Jul 16 10:50:12.461672 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:12.461650    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"
Jul 16 10:50:12.611752 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:12.611725    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6kb2\" (UniqueName: \"kubernetes.io/projected/73747b0d-6488-4c6f-a2a7-941c3dece66d-kube-api-access-b6kb2\") pod \"73747b0d-6488-4c6f-a2a7-941c3dece66d\" (UID: \"73747b0d-6488-4c6f-a2a7-941c3dece66d\") "
Jul 16 10:50:12.611902 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:12.611834    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73747b0d-6488-4c6f-a2a7-941c3dece66d-extensions-socket-volume\") pod \"73747b0d-6488-4c6f-a2a7-941c3dece66d\" (UID: \"73747b0d-6488-4c6f-a2a7-941c3dece66d\") "
Jul 16 10:50:12.612203 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:12.612178    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73747b0d-6488-4c6f-a2a7-941c3dece66d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "73747b0d-6488-4c6f-a2a7-941c3dece66d" (UID: "73747b0d-6488-4c6f-a2a7-941c3dece66d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 10:50:12.613764 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:12.613719    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73747b0d-6488-4c6f-a2a7-941c3dece66d-kube-api-access-b6kb2" (OuterVolumeSpecName: "kube-api-access-b6kb2") pod "73747b0d-6488-4c6f-a2a7-941c3dece66d" (UID: "73747b0d-6488-4c6f-a2a7-941c3dece66d"). InnerVolumeSpecName "kube-api-access-b6kb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 10:50:12.713297 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:12.713262    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73747b0d-6488-4c6f-a2a7-941c3dece66d-extensions-socket-volume\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:50:12.713297 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:12.713301    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6kb2\" (UniqueName: \"kubernetes.io/projected/73747b0d-6488-4c6f-a2a7-941c3dece66d-kube-api-access-b6kb2\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:50:13.375966 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:13.375939    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"
Jul 16 10:50:13.376428 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:13.375984    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v" event={"ID":"73747b0d-6488-4c6f-a2a7-941c3dece66d","Type":"ContainerDied","Data":"af057570cc616bb4e5fadcd70f20d916f6988dd66ffbffddd19ef791d729d503"}
Jul 16 10:50:13.376428 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:13.376054    2576 scope.go:117] "RemoveContainer" containerID="31646e036dee29217c2b718952e9b9980287d933e457df08b524b97194544864"
Jul 16 10:50:13.397919 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:13.397896    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"]
Jul 16 10:50:13.401941 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:13.401921    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hgf9v"]
Jul 16 10:50:13.801237 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:50:13.801160    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73747b0d-6488-4c6f-a2a7-941c3dece66d" path="/var/lib/kubelet/pods/73747b0d-6488-4c6f-a2a7-941c3dece66d/volumes"
Jul 16 10:51:10.758243 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.758210    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7"]
Jul 16 10:51:10.758696 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.758562    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a" containerName="authorino"
Jul 16 10:51:10.758696 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.758572    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a" containerName="authorino"
Jul 16 10:51:10.758696 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.758589    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73747b0d-6488-4c6f-a2a7-941c3dece66d" containerName="manager"
Jul 16 10:51:10.758696 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.758594    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="73747b0d-6488-4c6f-a2a7-941c3dece66d" containerName="manager"
Jul 16 10:51:10.758696 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.758648    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1a79b79-61e6-4e4c-8e4c-17e45fa2bf3a" containerName="authorino"
Jul 16 10:51:10.758696 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.758658    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="73747b0d-6488-4c6f-a2a7-941c3dece66d" containerName="manager"
Jul 16 10:51:10.761688 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.761669    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7"
Jul 16 10:51:10.764255 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.764229    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 10:51:10.764368 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.764259    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-g4tqf\""
Jul 16 10:51:10.764368 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.764232    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 10:51:10.776438 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.776418    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7"]
Jul 16 10:51:10.903794 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.903739    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/976132e9-6e42-4bf7-ae7f-f77ab99b3414-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-98pr7\" (UID: \"976132e9-6e42-4bf7-ae7f-f77ab99b3414\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7"
Jul 16 10:51:10.903952 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:10.903923    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qljhx\" (UniqueName: \"kubernetes.io/projected/976132e9-6e42-4bf7-ae7f-f77ab99b3414-kube-api-access-qljhx\") pod \"kuadrant-operator-controller-manager-86878c769f-98pr7\" (UID: \"976132e9-6e42-4bf7-ae7f-f77ab99b3414\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7"
Jul 16 10:51:11.005103 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:11.005055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/976132e9-6e42-4bf7-ae7f-f77ab99b3414-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-98pr7\" (UID: \"976132e9-6e42-4bf7-ae7f-f77ab99b3414\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7"
Jul 16 10:51:11.005288 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:11.005172    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qljhx\" (UniqueName: \"kubernetes.io/projected/976132e9-6e42-4bf7-ae7f-f77ab99b3414-kube-api-access-qljhx\") pod \"kuadrant-operator-controller-manager-86878c769f-98pr7\" (UID: \"976132e9-6e42-4bf7-ae7f-f77ab99b3414\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7"
Jul 16 10:51:11.005419 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:11.005399    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/976132e9-6e42-4bf7-ae7f-f77ab99b3414-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-98pr7\" (UID: \"976132e9-6e42-4bf7-ae7f-f77ab99b3414\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7"
Jul 16 10:51:11.014353 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:11.014292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qljhx\" (UniqueName: \"kubernetes.io/projected/976132e9-6e42-4bf7-ae7f-f77ab99b3414-kube-api-access-qljhx\") pod \"kuadrant-operator-controller-manager-86878c769f-98pr7\" (UID: \"976132e9-6e42-4bf7-ae7f-f77ab99b3414\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7"
Jul 16 10:51:11.072135 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:11.072111    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7"
Jul 16 10:51:11.207553 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:11.207520    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7"]
Jul 16 10:51:11.210850 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:51:11.210823    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod976132e9_6e42_4bf7_ae7f_f77ab99b3414.slice/crio-6392a5880ec728a5cfa50e02b272e58742f52a84957b74eafe581e121bd02a78 WatchSource:0}: Error finding container 6392a5880ec728a5cfa50e02b272e58742f52a84957b74eafe581e121bd02a78: Status 404 returned error can't find the container with id 6392a5880ec728a5cfa50e02b272e58742f52a84957b74eafe581e121bd02a78
Jul 16 10:51:11.625471 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:11.625435    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7" event={"ID":"976132e9-6e42-4bf7-ae7f-f77ab99b3414","Type":"ContainerStarted","Data":"bc716571c63e25a0d4a06a60026e723e5e078737d8aed95dee5899818ad1daae"}
Jul 16 10:51:11.625650 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:11.625497    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7" event={"ID":"976132e9-6e42-4bf7-ae7f-f77ab99b3414","Type":"ContainerStarted","Data":"6392a5880ec728a5cfa50e02b272e58742f52a84957b74eafe581e121bd02a78"}
Jul 16 10:51:11.625650 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:11.625526    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7"
Jul 16 10:51:11.646184 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:11.646127    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7" podStartSLOduration=1.646108956 podStartE2EDuration="1.646108956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:51:11.645697843 +0000 UTC m=+1126.382733116" watchObservedRunningTime="2026-07-16 10:51:11.646108956 +0000 UTC m=+1126.383144227"
Jul 16 10:51:22.631671 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:51:22.631638    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-98pr7"
Jul 16 10:52:27.866933 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:52:27.866907    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:52:27.870245 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:52:27.870225    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:57:27.892811 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:27.892689    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:57:27.898680 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:27.898662    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 10:57:48.827925 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:48.827832    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"]
Jul 16 10:57:48.831823 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:48.831800    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:48.834529 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:48.834507    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-eb5cfebe-serving-cert\""
Jul 16 10:57:48.834647 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:48.834516    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 10:57:48.835229 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:48.835211    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kn7cc\""
Jul 16 10:57:48.835372 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:48.835230    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 10:57:48.849482 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:48.849463    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"]
Jul 16 10:57:48.919047 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:48.919015    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72bh6\" (UniqueName: \"kubernetes.io/projected/1f7e87bb-64bb-4b77-a835-4f811187312b-kube-api-access-72bh6\") pod \"maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:48.919183 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:48.919053    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f7e87bb-64bb-4b77-a835-4f811187312b-openshift-service-ca\") pod \"maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:48.919183 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:48.919164    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f7e87bb-64bb-4b77-a835-4f811187312b-maas-api-tls\") pod \"maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:49.019894 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:49.019871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72bh6\" (UniqueName: \"kubernetes.io/projected/1f7e87bb-64bb-4b77-a835-4f811187312b-kube-api-access-72bh6\") pod \"maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:49.020006 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:49.019905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f7e87bb-64bb-4b77-a835-4f811187312b-openshift-service-ca\") pod \"maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:49.020006 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:49.019977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f7e87bb-64bb-4b77-a835-4f811187312b-maas-api-tls\") pod \"maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:49.020097 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:57:49.020083    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-serving-cert: secret "maas-api-e2e-ait-eb5cfebe-serving-cert" not found
Jul 16 10:57:49.020158 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:57:49.020147    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f7e87bb-64bb-4b77-a835-4f811187312b-maas-api-tls podName:1f7e87bb-64bb-4b77-a835-4f811187312b nodeName:}" failed. No retries permitted until 2026-07-16 10:57:49.520128987 +0000 UTC m=+1524.257164250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1f7e87bb-64bb-4b77-a835-4f811187312b-maas-api-tls") pod "maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4" (UID: "1f7e87bb-64bb-4b77-a835-4f811187312b") : secret "maas-api-e2e-ait-eb5cfebe-serving-cert" not found
Jul 16 10:57:49.020554 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:49.020530    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f7e87bb-64bb-4b77-a835-4f811187312b-openshift-service-ca\") pod \"maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:49.029076 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:49.029033    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72bh6\" (UniqueName: \"kubernetes.io/projected/1f7e87bb-64bb-4b77-a835-4f811187312b-kube-api-access-72bh6\") pod \"maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:49.523624 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:49.523592    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f7e87bb-64bb-4b77-a835-4f811187312b-maas-api-tls\") pod \"maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:49.523834 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:57:49.523791    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-serving-cert: secret "maas-api-e2e-ait-eb5cfebe-serving-cert" not found
Jul 16 10:57:49.523906 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:57:49.523862    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f7e87bb-64bb-4b77-a835-4f811187312b-maas-api-tls podName:1f7e87bb-64bb-4b77-a835-4f811187312b nodeName:}" failed. No retries permitted until 2026-07-16 10:57:50.523842779 +0000 UTC m=+1525.260878030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1f7e87bb-64bb-4b77-a835-4f811187312b-maas-api-tls") pod "maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4" (UID: "1f7e87bb-64bb-4b77-a835-4f811187312b") : secret "maas-api-e2e-ait-eb5cfebe-serving-cert" not found
Jul 16 10:57:50.532526 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:50.532497    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f7e87bb-64bb-4b77-a835-4f811187312b-maas-api-tls\") pod \"maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:50.535055 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:50.535020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f7e87bb-64bb-4b77-a835-4f811187312b-maas-api-tls\") pod \"maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:50.642923 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:50.642891    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:50.778569 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:50.778545    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"]
Jul 16 10:57:50.780212 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:57:50.780188    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f7e87bb_64bb_4b77_a835_4f811187312b.slice/crio-8b679d0fa3971a28e3d6c7a60a60250a47b02314e73122b90852300e1b52ddb3 WatchSource:0}: Error finding container 8b679d0fa3971a28e3d6c7a60a60250a47b02314e73122b90852300e1b52ddb3: Status 404 returned error can't find the container with id 8b679d0fa3971a28e3d6c7a60a60250a47b02314e73122b90852300e1b52ddb3
Jul 16 10:57:50.781552 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:50.781521    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 10:57:51.372980 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:51.372922    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4" event={"ID":"1f7e87bb-64bb-4b77-a835-4f811187312b","Type":"ContainerStarted","Data":"8b679d0fa3971a28e3d6c7a60a60250a47b02314e73122b90852300e1b52ddb3"}
Jul 16 10:57:53.383708 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:53.383677    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4" event={"ID":"1f7e87bb-64bb-4b77-a835-4f811187312b","Type":"ContainerStarted","Data":"07021957dab662e35e78a87d7c4f18fe87b07eb00860dba35e16a45a3ab8a54f"}
Jul 16 10:57:53.384131 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:53.383796    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:57:53.404284 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:53.404236    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4" podStartSLOduration=3.125987758 podStartE2EDuration="5.404218733s" totalImagesPullingTime="2.278230975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:57:53.403665938 +0000 UTC m=+1528.140701210" watchObservedRunningTime="2026-07-16 10:57:53.404218733 +0000 UTC m=+1528.141254005"
Jul 16 10:57:59.392739 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:57:59.392708    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:58:01.296397 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.296368    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"]
Jul 16 10:58:01.296779 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.296542    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4" podUID="1f7e87bb-64bb-4b77-a835-4f811187312b" containerName="maas-api" containerID="cri-o://07021957dab662e35e78a87d7c4f18fe87b07eb00860dba35e16a45a3ab8a54f" gracePeriod=30
Jul 16 10:58:01.421691 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.421658    2576 generic.go:356] "Generic (PLEG): container finished" podID="1f7e87bb-64bb-4b77-a835-4f811187312b" containerID="07021957dab662e35e78a87d7c4f18fe87b07eb00860dba35e16a45a3ab8a54f" exitCode=0
Jul 16 10:58:01.421691 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.421702    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4" event={"ID":"1f7e87bb-64bb-4b77-a835-4f811187312b","Type":"ContainerDied","Data":"07021957dab662e35e78a87d7c4f18fe87b07eb00860dba35e16a45a3ab8a54f"}
Jul 16 10:58:01.544591 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.544565    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:58:01.625041 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.625016    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f7e87bb-64bb-4b77-a835-4f811187312b-openshift-service-ca\") pod \"1f7e87bb-64bb-4b77-a835-4f811187312b\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") "
Jul 16 10:58:01.625182 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.625049    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72bh6\" (UniqueName: \"kubernetes.io/projected/1f7e87bb-64bb-4b77-a835-4f811187312b-kube-api-access-72bh6\") pod \"1f7e87bb-64bb-4b77-a835-4f811187312b\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") "
Jul 16 10:58:01.625182 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.625079    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f7e87bb-64bb-4b77-a835-4f811187312b-maas-api-tls\") pod \"1f7e87bb-64bb-4b77-a835-4f811187312b\" (UID: \"1f7e87bb-64bb-4b77-a835-4f811187312b\") "
Jul 16 10:58:01.625427 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.625409    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f7e87bb-64bb-4b77-a835-4f811187312b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1f7e87bb-64bb-4b77-a835-4f811187312b" (UID: "1f7e87bb-64bb-4b77-a835-4f811187312b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 10:58:01.627276 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.627244    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f7e87bb-64bb-4b77-a835-4f811187312b-kube-api-access-72bh6" (OuterVolumeSpecName: "kube-api-access-72bh6") pod "1f7e87bb-64bb-4b77-a835-4f811187312b" (UID: "1f7e87bb-64bb-4b77-a835-4f811187312b"). InnerVolumeSpecName "kube-api-access-72bh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 10:58:01.627398 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.627350    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f7e87bb-64bb-4b77-a835-4f811187312b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1f7e87bb-64bb-4b77-a835-4f811187312b" (UID: "1f7e87bb-64bb-4b77-a835-4f811187312b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 10:58:01.726384 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.726357    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f7e87bb-64bb-4b77-a835-4f811187312b-openshift-service-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:58:01.726384 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.726381    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72bh6\" (UniqueName: \"kubernetes.io/projected/1f7e87bb-64bb-4b77-a835-4f811187312b-kube-api-access-72bh6\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:58:01.726537 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:01.726391    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f7e87bb-64bb-4b77-a835-4f811187312b-maas-api-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:58:02.427236 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:02.427198    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4" event={"ID":"1f7e87bb-64bb-4b77-a835-4f811187312b","Type":"ContainerDied","Data":"8b679d0fa3971a28e3d6c7a60a60250a47b02314e73122b90852300e1b52ddb3"}
Jul 16 10:58:02.427688 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:02.427259    2576 scope.go:117] "RemoveContainer" containerID="07021957dab662e35e78a87d7c4f18fe87b07eb00860dba35e16a45a3ab8a54f"
Jul 16 10:58:02.427688 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:02.427211    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"
Jul 16 10:58:02.447124 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:02.447099    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"]
Jul 16 10:58:02.452441 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:02.452420    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb5cfebe-cd5c8bcd7-snkt4"]
Jul 16 10:58:03.800798 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:03.800763    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f7e87bb-64bb-4b77-a835-4f811187312b" path="/var/lib/kubelet/pods/1f7e87bb-64bb-4b77-a835-4f811187312b/volumes"
Jul 16 10:58:08.187200 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.187169    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"]
Jul 16 10:58:08.187724 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.187702    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f7e87bb-64bb-4b77-a835-4f811187312b" containerName="maas-api"
Jul 16 10:58:08.187724 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.187723    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f7e87bb-64bb-4b77-a835-4f811187312b" containerName="maas-api"
Jul 16 10:58:08.187944 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.187852    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f7e87bb-64bb-4b77-a835-4f811187312b" containerName="maas-api"
Jul 16 10:58:08.193930 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.193913    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:08.199149 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.199127    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-3eb28edb-serving-cert\""
Jul 16 10:58:08.199276 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.199195    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 10:58:08.199459 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.199441    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kn7cc\""
Jul 16 10:58:08.199593 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.199562    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 10:58:08.204352 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.204331    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"]
Jul 16 10:58:08.279706 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.279684    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trb25\" (UniqueName: \"kubernetes.io/projected/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-kube-api-access-trb25\") pod \"maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf\" (UID: \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:08.279849 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.279733    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-openshift-service-ca\") pod \"maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf\" (UID: \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:08.279849 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.279835    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-maas-api-tls\") pod \"maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf\" (UID: \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:08.380849 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.380821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-openshift-service-ca\") pod \"maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf\" (UID: \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:08.380959 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.380882    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-maas-api-tls\") pod \"maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf\" (UID: \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:08.380959 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.380933    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trb25\" (UniqueName: \"kubernetes.io/projected/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-kube-api-access-trb25\") pod \"maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf\" (UID: \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:08.381069 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:58:08.381027    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-serving-cert: secret "maas-api-e2e-migrate-3eb28edb-serving-cert" not found
Jul 16 10:58:08.381119 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:58:08.381100    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-maas-api-tls podName:62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2 nodeName:}" failed. No retries permitted until 2026-07-16 10:58:08.881081726 +0000 UTC m=+1543.618116998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-maas-api-tls") pod "maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf" (UID: "62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2") : secret "maas-api-e2e-migrate-3eb28edb-serving-cert" not found
Jul 16 10:58:08.381448 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.381430    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-openshift-service-ca\") pod \"maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf\" (UID: \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:08.391239 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.391212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trb25\" (UniqueName: \"kubernetes.io/projected/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-kube-api-access-trb25\") pod \"maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf\" (UID: \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:08.884003 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.883967    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-maas-api-tls\") pod \"maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf\" (UID: \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:08.886520 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:08.886496    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-maas-api-tls\") pod \"maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf\" (UID: \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:09.105056 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:09.105025    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:09.239399 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:09.239366    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"]
Jul 16 10:58:09.243075 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:58:09.243051    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c26f82_ab9c_4ea9_94f6_4afb38e4f2d2.slice/crio-e3e1ab62a75e693d3681b4379643ac6b5644c52c4d5512f1cbba24e852fa3041 WatchSource:0}: Error finding container e3e1ab62a75e693d3681b4379643ac6b5644c52c4d5512f1cbba24e852fa3041: Status 404 returned error can't find the container with id e3e1ab62a75e693d3681b4379643ac6b5644c52c4d5512f1cbba24e852fa3041
Jul 16 10:58:09.462809 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:09.462723    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf" event={"ID":"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2","Type":"ContainerStarted","Data":"e3e1ab62a75e693d3681b4379643ac6b5644c52c4d5512f1cbba24e852fa3041"}
Jul 16 10:58:10.468405 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:10.468375    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf" event={"ID":"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2","Type":"ContainerStarted","Data":"f0094ddfeace582635d9d77f6d597178d36aad6a2413c95946498be7eb4a872f"}
Jul 16 10:58:10.468840 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:10.468498    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:10.488074 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:10.488031    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf" podStartSLOduration=2.144239618 podStartE2EDuration="2.4880175s" totalImagesPullingTime="343.777882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:58:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:58:10.486602834 +0000 UTC m=+1545.223638104" watchObservedRunningTime="2026-07-16 10:58:10.4880175 +0000 UTC m=+1545.225052771"
Jul 16 10:58:16.478213 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:16.478182    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:19.061032 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.060997    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"]
Jul 16 10:58:19.061481 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.061245    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf" podUID="62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2" containerName="maas-api" containerID="cri-o://f0094ddfeace582635d9d77f6d597178d36aad6a2413c95946498be7eb4a872f" gracePeriod=30
Jul 16 10:58:19.305326 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.305303    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:19.374654 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.374629    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trb25\" (UniqueName: \"kubernetes.io/projected/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-kube-api-access-trb25\") pod \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\" (UID: \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\") "
Jul 16 10:58:19.374795 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.374697    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-maas-api-tls\") pod \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\" (UID: \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\") "
Jul 16 10:58:19.374795 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.374780    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-openshift-service-ca\") pod \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\" (UID: \"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2\") "
Jul 16 10:58:19.375123 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.375098    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2" (UID: "62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 10:58:19.376652 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.376628    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2" (UID: "62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 10:58:19.376773 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.376687    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-kube-api-access-trb25" (OuterVolumeSpecName: "kube-api-access-trb25") pod "62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2" (UID: "62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2"). InnerVolumeSpecName "kube-api-access-trb25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 10:58:19.475788 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.475761    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-openshift-service-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:58:19.475788 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.475785    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trb25\" (UniqueName: \"kubernetes.io/projected/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-kube-api-access-trb25\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:58:19.475928 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.475796    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2-maas-api-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:58:19.507454 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.507427    2576 generic.go:356] "Generic (PLEG): container finished" podID="62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2" containerID="f0094ddfeace582635d9d77f6d597178d36aad6a2413c95946498be7eb4a872f" exitCode=0
Jul 16 10:58:19.507541 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.507477    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"
Jul 16 10:58:19.507541 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.507517    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf" event={"ID":"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2","Type":"ContainerDied","Data":"f0094ddfeace582635d9d77f6d597178d36aad6a2413c95946498be7eb4a872f"}
Jul 16 10:58:19.507616 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.507571    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf" event={"ID":"62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2","Type":"ContainerDied","Data":"e3e1ab62a75e693d3681b4379643ac6b5644c52c4d5512f1cbba24e852fa3041"}
Jul 16 10:58:19.507616 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.507594    2576 scope.go:117] "RemoveContainer" containerID="f0094ddfeace582635d9d77f6d597178d36aad6a2413c95946498be7eb4a872f"
Jul 16 10:58:19.516590 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.516561    2576 scope.go:117] "RemoveContainer" containerID="f0094ddfeace582635d9d77f6d597178d36aad6a2413c95946498be7eb4a872f"
Jul 16 10:58:19.516855 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:58:19.516836    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0094ddfeace582635d9d77f6d597178d36aad6a2413c95946498be7eb4a872f\": container with ID starting with f0094ddfeace582635d9d77f6d597178d36aad6a2413c95946498be7eb4a872f not found: ID does not exist" containerID="f0094ddfeace582635d9d77f6d597178d36aad6a2413c95946498be7eb4a872f"
Jul 16 10:58:19.516911 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.516863    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0094ddfeace582635d9d77f6d597178d36aad6a2413c95946498be7eb4a872f"} err="failed to get container status \"f0094ddfeace582635d9d77f6d597178d36aad6a2413c95946498be7eb4a872f\": rpc error: code = NotFound desc = could not find container \"f0094ddfeace582635d9d77f6d597178d36aad6a2413c95946498be7eb4a872f\": container with ID starting with f0094ddfeace582635d9d77f6d597178d36aad6a2413c95946498be7eb4a872f not found: ID does not exist"
Jul 16 10:58:19.528435 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.528411    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"]
Jul 16 10:58:19.530668 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.530644    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3eb28edb-5c47b7f686-c5phf"]
Jul 16 10:58:19.802205 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:19.802134    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2" path="/var/lib/kubelet/pods/62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2/volumes"
Jul 16 10:58:26.357815 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.357782    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"]
Jul 16 10:58:26.358266 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.358143    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2" containerName="maas-api"
Jul 16 10:58:26.358266 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.358154    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2" containerName="maas-api"
Jul 16 10:58:26.358266 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.358216    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="62c26f82-ab9c-4ea9-94f6-4afb38e4f2d2" containerName="maas-api"
Jul 16 10:58:26.361056 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.361035    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:26.363774 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.363731    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kn7cc\""
Jul 16 10:58:26.363774 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.363758    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-95aa3181-serving-cert\""
Jul 16 10:58:26.364294 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.364278    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 10:58:26.364584 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.364560    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 10:58:26.375335 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.375282    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"]
Jul 16 10:58:26.434284 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.434259    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/472e878a-7f8e-4970-9bd0-da28f0ba6671-maas-api-tls\") pod \"maas-api-e2e-derive-95aa3181-98d79b886-wv9dx\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:26.434376 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.434314    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/472e878a-7f8e-4970-9bd0-da28f0ba6671-openshift-service-ca\") pod \"maas-api-e2e-derive-95aa3181-98d79b886-wv9dx\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:26.434418 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.434386    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl4rn\" (UniqueName: \"kubernetes.io/projected/472e878a-7f8e-4970-9bd0-da28f0ba6671-kube-api-access-zl4rn\") pod \"maas-api-e2e-derive-95aa3181-98d79b886-wv9dx\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:26.535509 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.535482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl4rn\" (UniqueName: \"kubernetes.io/projected/472e878a-7f8e-4970-9bd0-da28f0ba6671-kube-api-access-zl4rn\") pod \"maas-api-e2e-derive-95aa3181-98d79b886-wv9dx\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:26.535622 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.535525    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/472e878a-7f8e-4970-9bd0-da28f0ba6671-maas-api-tls\") pod \"maas-api-e2e-derive-95aa3181-98d79b886-wv9dx\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:26.535622 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.535565    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/472e878a-7f8e-4970-9bd0-da28f0ba6671-openshift-service-ca\") pod \"maas-api-e2e-derive-95aa3181-98d79b886-wv9dx\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:26.535718 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:58:26.535705    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-serving-cert: secret "maas-api-e2e-derive-95aa3181-serving-cert" not found
Jul 16 10:58:26.535819 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:58:26.535804    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/472e878a-7f8e-4970-9bd0-da28f0ba6671-maas-api-tls podName:472e878a-7f8e-4970-9bd0-da28f0ba6671 nodeName:}" failed. No retries permitted until 2026-07-16 10:58:27.035783976 +0000 UTC m=+1561.772819241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/472e878a-7f8e-4970-9bd0-da28f0ba6671-maas-api-tls") pod "maas-api-e2e-derive-95aa3181-98d79b886-wv9dx" (UID: "472e878a-7f8e-4970-9bd0-da28f0ba6671") : secret "maas-api-e2e-derive-95aa3181-serving-cert" not found
Jul 16 10:58:26.536171 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.536154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/472e878a-7f8e-4970-9bd0-da28f0ba6671-openshift-service-ca\") pod \"maas-api-e2e-derive-95aa3181-98d79b886-wv9dx\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:26.547527 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:26.547505    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl4rn\" (UniqueName: \"kubernetes.io/projected/472e878a-7f8e-4970-9bd0-da28f0ba6671-kube-api-access-zl4rn\") pod \"maas-api-e2e-derive-95aa3181-98d79b886-wv9dx\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:27.039354 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:27.039319    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/472e878a-7f8e-4970-9bd0-da28f0ba6671-maas-api-tls\") pod \"maas-api-e2e-derive-95aa3181-98d79b886-wv9dx\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:27.039510 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:58:27.039468    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-serving-cert: secret "maas-api-e2e-derive-95aa3181-serving-cert" not found
Jul 16 10:58:27.039570 ip-10-0-128-213 kubenswrapper[2576]: E0716 10:58:27.039557    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/472e878a-7f8e-4970-9bd0-da28f0ba6671-maas-api-tls podName:472e878a-7f8e-4970-9bd0-da28f0ba6671 nodeName:}" failed. No retries permitted until 2026-07-16 10:58:28.039535726 +0000 UTC m=+1562.776570977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/472e878a-7f8e-4970-9bd0-da28f0ba6671-maas-api-tls") pod "maas-api-e2e-derive-95aa3181-98d79b886-wv9dx" (UID: "472e878a-7f8e-4970-9bd0-da28f0ba6671") : secret "maas-api-e2e-derive-95aa3181-serving-cert" not found
Jul 16 10:58:28.050137 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:28.050102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/472e878a-7f8e-4970-9bd0-da28f0ba6671-maas-api-tls\") pod \"maas-api-e2e-derive-95aa3181-98d79b886-wv9dx\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:28.052404 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:28.052383    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/472e878a-7f8e-4970-9bd0-da28f0ba6671-maas-api-tls\") pod \"maas-api-e2e-derive-95aa3181-98d79b886-wv9dx\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:28.171644 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:28.171613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:28.306495 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:28.306469    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"]
Jul 16 10:58:28.308002 ip-10-0-128-213 kubenswrapper[2576]: W0716 10:58:28.307970    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod472e878a_7f8e_4970_9bd0_da28f0ba6671.slice/crio-16a2a90a31ee51278075d07c1abe4d00acefd29c1a4658743d9773dadbc8b2a6 WatchSource:0}: Error finding container 16a2a90a31ee51278075d07c1abe4d00acefd29c1a4658743d9773dadbc8b2a6: Status 404 returned error can't find the container with id 16a2a90a31ee51278075d07c1abe4d00acefd29c1a4658743d9773dadbc8b2a6
Jul 16 10:58:28.554438 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:28.554405    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx" event={"ID":"472e878a-7f8e-4970-9bd0-da28f0ba6671","Type":"ContainerStarted","Data":"16a2a90a31ee51278075d07c1abe4d00acefd29c1a4658743d9773dadbc8b2a6"}
Jul 16 10:58:29.562439 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:29.562409    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx" event={"ID":"472e878a-7f8e-4970-9bd0-da28f0ba6671","Type":"ContainerStarted","Data":"ea8e86edf2e38bbf01c8d9daff4774da4ff3a8c816ab0f5248dc8474208f160a"}
Jul 16 10:58:29.562814 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:29.562546    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:29.586397 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:29.586350    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx" podStartSLOduration=3.287388629 podStartE2EDuration="3.58633425s" totalImagesPullingTime="298.945621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:58:29.58555104 +0000 UTC m=+1564.322586324" watchObservedRunningTime="2026-07-16 10:58:29.58633425 +0000 UTC m=+1564.323369517"
Jul 16 10:58:35.572144 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:35.572119    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:37.389827 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.389791    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"]
Jul 16 10:58:37.390284 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.390009    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx" podUID="472e878a-7f8e-4970-9bd0-da28f0ba6671" containerName="maas-api" containerID="cri-o://ea8e86edf2e38bbf01c8d9daff4774da4ff3a8c816ab0f5248dc8474208f160a" gracePeriod=30
Jul 16 10:58:37.600791 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.600734    2576 generic.go:356] "Generic (PLEG): container finished" podID="472e878a-7f8e-4970-9bd0-da28f0ba6671" containerID="ea8e86edf2e38bbf01c8d9daff4774da4ff3a8c816ab0f5248dc8474208f160a" exitCode=0
Jul 16 10:58:37.600910 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.600812    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx" event={"ID":"472e878a-7f8e-4970-9bd0-da28f0ba6671","Type":"ContainerDied","Data":"ea8e86edf2e38bbf01c8d9daff4774da4ff3a8c816ab0f5248dc8474208f160a"}
Jul 16 10:58:37.629375 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.629347    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:37.729333 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.729268    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/472e878a-7f8e-4970-9bd0-da28f0ba6671-maas-api-tls\") pod \"472e878a-7f8e-4970-9bd0-da28f0ba6671\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") "
Jul 16 10:58:37.729333 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.729306    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zl4rn\" (UniqueName: \"kubernetes.io/projected/472e878a-7f8e-4970-9bd0-da28f0ba6671-kube-api-access-zl4rn\") pod \"472e878a-7f8e-4970-9bd0-da28f0ba6671\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") "
Jul 16 10:58:37.729333 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.729331    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/472e878a-7f8e-4970-9bd0-da28f0ba6671-openshift-service-ca\") pod \"472e878a-7f8e-4970-9bd0-da28f0ba6671\" (UID: \"472e878a-7f8e-4970-9bd0-da28f0ba6671\") "
Jul 16 10:58:37.729709 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.729683    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/472e878a-7f8e-4970-9bd0-da28f0ba6671-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "472e878a-7f8e-4970-9bd0-da28f0ba6671" (UID: "472e878a-7f8e-4970-9bd0-da28f0ba6671"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 10:58:37.731343 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.731313    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/472e878a-7f8e-4970-9bd0-da28f0ba6671-kube-api-access-zl4rn" (OuterVolumeSpecName: "kube-api-access-zl4rn") pod "472e878a-7f8e-4970-9bd0-da28f0ba6671" (UID: "472e878a-7f8e-4970-9bd0-da28f0ba6671"). InnerVolumeSpecName "kube-api-access-zl4rn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 10:58:37.731423 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.731350    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/472e878a-7f8e-4970-9bd0-da28f0ba6671-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "472e878a-7f8e-4970-9bd0-da28f0ba6671" (UID: "472e878a-7f8e-4970-9bd0-da28f0ba6671"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 10:58:37.830125 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.830099    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/472e878a-7f8e-4970-9bd0-da28f0ba6671-maas-api-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:58:37.830125 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.830122    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zl4rn\" (UniqueName: \"kubernetes.io/projected/472e878a-7f8e-4970-9bd0-da28f0ba6671-kube-api-access-zl4rn\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:58:37.830255 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:37.830137    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/472e878a-7f8e-4970-9bd0-da28f0ba6671-openshift-service-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 10:58:38.606186 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:38.606150    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx" event={"ID":"472e878a-7f8e-4970-9bd0-da28f0ba6671","Type":"ContainerDied","Data":"16a2a90a31ee51278075d07c1abe4d00acefd29c1a4658743d9773dadbc8b2a6"}
Jul 16 10:58:38.606186 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:38.606175    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"
Jul 16 10:58:38.606655 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:38.606219    2576 scope.go:117] "RemoveContainer" containerID="ea8e86edf2e38bbf01c8d9daff4774da4ff3a8c816ab0f5248dc8474208f160a"
Jul 16 10:58:38.624389 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:38.624353    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"]
Jul 16 10:58:38.629152 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:38.629131    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-95aa3181-98d79b886-wv9dx"]
Jul 16 10:58:39.801606 ip-10-0-128-213 kubenswrapper[2576]: I0716 10:58:39.801563    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="472e878a-7f8e-4970-9bd0-da28f0ba6671" path="/var/lib/kubelet/pods/472e878a-7f8e-4970-9bd0-da28f0ba6671/volumes"
Jul 16 11:00:27.351346 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.351314    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"]
Jul 16 11:00:27.351907 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.351670    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="472e878a-7f8e-4970-9bd0-da28f0ba6671" containerName="maas-api"
Jul 16 11:00:27.351907 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.351682    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="472e878a-7f8e-4970-9bd0-da28f0ba6671" containerName="maas-api"
Jul 16 11:00:27.351907 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.351757    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="472e878a-7f8e-4970-9bd0-da28f0ba6671" containerName="maas-api"
Jul 16 11:00:27.354612 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.354591    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:27.357567 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.357542    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kn7cc\""
Jul 16 11:00:27.357710 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.357551    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 11:00:27.357710 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.357661    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3b9fb924-serving-cert\""
Jul 16 11:00:27.358415 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.358396    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 11:00:27.375382 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.375363    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"]
Jul 16 11:00:27.500076 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.500052    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-maas-api-tls\") pod \"maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64\" (UID: \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:27.500183 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.500088    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjpjb\" (UniqueName: \"kubernetes.io/projected/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-kube-api-access-gjpjb\") pod \"maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64\" (UID: \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:27.500183 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.500121    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-openshift-service-ca\") pod \"maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64\" (UID: \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:27.600990 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.600963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-maas-api-tls\") pod \"maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64\" (UID: \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:27.601080 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.601003    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjpjb\" (UniqueName: \"kubernetes.io/projected/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-kube-api-access-gjpjb\") pod \"maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64\" (UID: \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:27.601080 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.601052    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-openshift-service-ca\") pod \"maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64\" (UID: \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:27.601732 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.601674    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-openshift-service-ca\") pod \"maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64\" (UID: \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:27.603376 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.603358    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-maas-api-tls\") pod \"maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64\" (UID: \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:27.610470 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.610447    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjpjb\" (UniqueName: \"kubernetes.io/projected/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-kube-api-access-gjpjb\") pod \"maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64\" (UID: \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:27.666231 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.666210    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:27.800926 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:00:27.800900    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb1b9eb7_25a1_4eaa_800b_4f821d61f1dc.slice/crio-30d20d6c10deb5719a0159d4109d2abe51c37d85a6c8e5aa007501036971cd74 WatchSource:0}: Error finding container 30d20d6c10deb5719a0159d4109d2abe51c37d85a6c8e5aa007501036971cd74: Status 404 returned error can't find the container with id 30d20d6c10deb5719a0159d4109d2abe51c37d85a6c8e5aa007501036971cd74
Jul 16 11:00:27.803412 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:27.802888    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"]
Jul 16 11:00:28.085897 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:28.085863    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64" event={"ID":"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc","Type":"ContainerStarted","Data":"30d20d6c10deb5719a0159d4109d2abe51c37d85a6c8e5aa007501036971cd74"}
Jul 16 11:00:29.091847 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:29.091762    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64" event={"ID":"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc","Type":"ContainerStarted","Data":"78f4cb2fcb80687bb50ca8b1435d8191132eef1f1b91f79ae7acd2e6ef30d297"}
Jul 16 11:00:29.092213 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:29.091881    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:29.113180 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:29.113126    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64" podStartSLOduration=1.239059274 podStartE2EDuration="2.113109726s" totalImagesPullingTime="874.050452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:00:29.111196146 +0000 UTC m=+1683.848231419" watchObservedRunningTime="2026-07-16 11:00:29.113109726 +0000 UTC m=+1683.850145016"
Jul 16 11:00:35.101967 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:35.101935    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:47.384239 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:47.384167    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"]
Jul 16 11:00:47.384738 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:47.384357    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64" podUID="bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc" containerName="maas-api" containerID="cri-o://78f4cb2fcb80687bb50ca8b1435d8191132eef1f1b91f79ae7acd2e6ef30d297" gracePeriod=30
Jul 16 11:00:47.627965 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:47.627943    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:47.666599 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:47.666540    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-maas-api-tls\") pod \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\" (UID: \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\") "
Jul 16 11:00:47.666725 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:47.666618    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-openshift-service-ca\") pod \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\" (UID: \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\") "
Jul 16 11:00:47.666725 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:47.666645    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjpjb\" (UniqueName: \"kubernetes.io/projected/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-kube-api-access-gjpjb\") pod \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\" (UID: \"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc\") "
Jul 16 11:00:47.667006 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:47.666974    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc" (UID: "bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:00:47.668682 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:47.668655    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-kube-api-access-gjpjb" (OuterVolumeSpecName: "kube-api-access-gjpjb") pod "bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc" (UID: "bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc"). InnerVolumeSpecName "kube-api-access-gjpjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:00:47.668792 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:47.668677    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc" (UID: "bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:00:47.767489 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:47.767459    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-openshift-service-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:00:47.767489 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:47.767485    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjpjb\" (UniqueName: \"kubernetes.io/projected/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-kube-api-access-gjpjb\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:00:47.767489 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:47.767496    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc-maas-api-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:00:48.179137 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:48.179104    2576 generic.go:356] "Generic (PLEG): container finished" podID="bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc" containerID="78f4cb2fcb80687bb50ca8b1435d8191132eef1f1b91f79ae7acd2e6ef30d297" exitCode=0
Jul 16 11:00:48.179321 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:48.179154    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"
Jul 16 11:00:48.179321 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:48.179183    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64" event={"ID":"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc","Type":"ContainerDied","Data":"78f4cb2fcb80687bb50ca8b1435d8191132eef1f1b91f79ae7acd2e6ef30d297"}
Jul 16 11:00:48.179321 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:48.179235    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64" event={"ID":"bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc","Type":"ContainerDied","Data":"30d20d6c10deb5719a0159d4109d2abe51c37d85a6c8e5aa007501036971cd74"}
Jul 16 11:00:48.179321 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:48.179257    2576 scope.go:117] "RemoveContainer" containerID="78f4cb2fcb80687bb50ca8b1435d8191132eef1f1b91f79ae7acd2e6ef30d297"
Jul 16 11:00:48.193246 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:48.193230    2576 scope.go:117] "RemoveContainer" containerID="78f4cb2fcb80687bb50ca8b1435d8191132eef1f1b91f79ae7acd2e6ef30d297"
Jul 16 11:00:48.193487 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:00:48.193462    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78f4cb2fcb80687bb50ca8b1435d8191132eef1f1b91f79ae7acd2e6ef30d297\": container with ID starting with 78f4cb2fcb80687bb50ca8b1435d8191132eef1f1b91f79ae7acd2e6ef30d297 not found: ID does not exist" containerID="78f4cb2fcb80687bb50ca8b1435d8191132eef1f1b91f79ae7acd2e6ef30d297"
Jul 16 11:00:48.193587 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:48.193491    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78f4cb2fcb80687bb50ca8b1435d8191132eef1f1b91f79ae7acd2e6ef30d297"} err="failed to get container status \"78f4cb2fcb80687bb50ca8b1435d8191132eef1f1b91f79ae7acd2e6ef30d297\": rpc error: code = NotFound desc = could not find container \"78f4cb2fcb80687bb50ca8b1435d8191132eef1f1b91f79ae7acd2e6ef30d297\": container with ID starting with 78f4cb2fcb80687bb50ca8b1435d8191132eef1f1b91f79ae7acd2e6ef30d297 not found: ID does not exist"
Jul 16 11:00:48.201860 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:48.201840    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"]
Jul 16 11:00:48.209396 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:48.209374    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3b9fb924-54f7579bf8-dcb64"]
Jul 16 11:00:49.801149 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:00:49.801115    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc" path="/var/lib/kubelet/pods/bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc/volumes"
Jul 16 11:02:12.933640 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:12.933605    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"]
Jul 16 11:02:12.934248 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:12.934144    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc" containerName="maas-api"
Jul 16 11:02:12.934248 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:12.934165    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc" containerName="maas-api"
Jul 16 11:02:12.934248 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:12.934248    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb1b9eb7-25a1-4eaa-800b-4f821d61f1dc" containerName="maas-api"
Jul 16 11:02:12.937580 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:12.937549    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:12.940302 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:12.940276    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 11:02:12.940440 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:12.940348    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kn7cc\""
Jul 16 11:02:12.940613 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:12.940593    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 11:02:12.941543 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:12.941509    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-5e601d-serving-cert\""
Jul 16 11:02:12.956461 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:12.953731    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"]
Jul 16 11:02:13.023816 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:13.023789    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1803d69-370b-4b11-bc3e-511e7a9fbc13-maas-api-tls\") pod \"maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99\" (UID: \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:13.023914 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:13.023852    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvhcq\" (UniqueName: \"kubernetes.io/projected/f1803d69-370b-4b11-bc3e-511e7a9fbc13-kube-api-access-hvhcq\") pod \"maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99\" (UID: \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:13.023914 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:13.023894    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1803d69-370b-4b11-bc3e-511e7a9fbc13-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99\" (UID: \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:13.124857 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:13.124832    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1803d69-370b-4b11-bc3e-511e7a9fbc13-maas-api-tls\") pod \"maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99\" (UID: \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:13.124955 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:13.124889    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvhcq\" (UniqueName: \"kubernetes.io/projected/f1803d69-370b-4b11-bc3e-511e7a9fbc13-kube-api-access-hvhcq\") pod \"maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99\" (UID: \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:13.124955 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:13.124930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1803d69-370b-4b11-bc3e-511e7a9fbc13-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99\" (UID: \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:13.125047 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:02:13.124967    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-serving-cert: secret "maas-api-e2e-inf-a-5e601d-serving-cert" not found
Jul 16 11:02:13.125047 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:02:13.125020    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1803d69-370b-4b11-bc3e-511e7a9fbc13-maas-api-tls podName:f1803d69-370b-4b11-bc3e-511e7a9fbc13 nodeName:}" failed. No retries permitted until 2026-07-16 11:02:13.625006771 +0000 UTC m=+1788.362042021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f1803d69-370b-4b11-bc3e-511e7a9fbc13-maas-api-tls") pod "maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99" (UID: "f1803d69-370b-4b11-bc3e-511e7a9fbc13") : secret "maas-api-e2e-inf-a-5e601d-serving-cert" not found
Jul 16 11:02:13.125510 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:13.125495    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1803d69-370b-4b11-bc3e-511e7a9fbc13-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99\" (UID: \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:13.135671 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:13.135639    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvhcq\" (UniqueName: \"kubernetes.io/projected/f1803d69-370b-4b11-bc3e-511e7a9fbc13-kube-api-access-hvhcq\") pod \"maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99\" (UID: \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:13.628929 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:13.628896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1803d69-370b-4b11-bc3e-511e7a9fbc13-maas-api-tls\") pod \"maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99\" (UID: \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:13.631255 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:13.631226    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1803d69-370b-4b11-bc3e-511e7a9fbc13-maas-api-tls\") pod \"maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99\" (UID: \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:13.849227 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:13.849196    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:13.980007 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:13.979981    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"]
Jul 16 11:02:13.982255 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:02:13.982228    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1803d69_370b_4b11_bc3e_511e7a9fbc13.slice/crio-6249bfb6ed4855e4dded8722aed2403727163800cee9cd65457026c381b5e738 WatchSource:0}: Error finding container 6249bfb6ed4855e4dded8722aed2403727163800cee9cd65457026c381b5e738: Status 404 returned error can't find the container with id 6249bfb6ed4855e4dded8722aed2403727163800cee9cd65457026c381b5e738
Jul 16 11:02:14.559838 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:14.559807    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99" event={"ID":"f1803d69-370b-4b11-bc3e-511e7a9fbc13","Type":"ContainerStarted","Data":"6249bfb6ed4855e4dded8722aed2403727163800cee9cd65457026c381b5e738"}
Jul 16 11:02:15.565870 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:15.565834    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99" event={"ID":"f1803d69-370b-4b11-bc3e-511e7a9fbc13","Type":"ContainerStarted","Data":"848f7ba9a4ad319aa2af888db14c037d46a2cdbeea880d728517d6e5563038e6"}
Jul 16 11:02:15.566226 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:15.565941    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:15.584260 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:15.584220    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99" podStartSLOduration=2.549232284 podStartE2EDuration="3.584206566s" totalImagesPullingTime="1.034974282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:02:15.582431096 +0000 UTC m=+1790.319466366" watchObservedRunningTime="2026-07-16 11:02:15.584206566 +0000 UTC m=+1790.321241866"
Jul 16 11:02:21.348175 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.348142    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"]
Jul 16 11:02:21.351813 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.351794    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:21.355450 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.355431    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-d156b9-serving-cert\""
Jul 16 11:02:21.383609 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.383585    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"]
Jul 16 11:02:21.393064 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.393043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7da81d0-65af-4bcf-8b9f-bc70b979898e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms\" (UID: \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:21.393164 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.393149    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7da81d0-65af-4bcf-8b9f-bc70b979898e-maas-api-tls\") pod \"maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms\" (UID: \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:21.393214 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.393199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7vfn\" (UniqueName: \"kubernetes.io/projected/b7da81d0-65af-4bcf-8b9f-bc70b979898e-kube-api-access-j7vfn\") pod \"maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms\" (UID: \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:21.494206 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.494174    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7da81d0-65af-4bcf-8b9f-bc70b979898e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms\" (UID: \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:21.494347 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.494281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7da81d0-65af-4bcf-8b9f-bc70b979898e-maas-api-tls\") pod \"maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms\" (UID: \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:21.494347 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.494324    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7vfn\" (UniqueName: \"kubernetes.io/projected/b7da81d0-65af-4bcf-8b9f-bc70b979898e-kube-api-access-j7vfn\") pod \"maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms\" (UID: \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:21.494459 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:02:21.494416    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-serving-cert: secret "maas-api-e2e-inf-b-d156b9-serving-cert" not found
Jul 16 11:02:21.494511 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:02:21.494481    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7da81d0-65af-4bcf-8b9f-bc70b979898e-maas-api-tls podName:b7da81d0-65af-4bcf-8b9f-bc70b979898e nodeName:}" failed. No retries permitted until 2026-07-16 11:02:21.994466129 +0000 UTC m=+1796.731501381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b7da81d0-65af-4bcf-8b9f-bc70b979898e-maas-api-tls") pod "maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms" (UID: "b7da81d0-65af-4bcf-8b9f-bc70b979898e") : secret "maas-api-e2e-inf-b-d156b9-serving-cert" not found
Jul 16 11:02:21.494818 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.494800    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7da81d0-65af-4bcf-8b9f-bc70b979898e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms\" (UID: \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:21.512068 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.512045    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7vfn\" (UniqueName: \"kubernetes.io/projected/b7da81d0-65af-4bcf-8b9f-bc70b979898e-kube-api-access-j7vfn\") pod \"maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms\" (UID: \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:21.575583 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.575563    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:02:21.999364 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:21.999325    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7da81d0-65af-4bcf-8b9f-bc70b979898e-maas-api-tls\") pod \"maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms\" (UID: \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:22.002026 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:22.002002    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7da81d0-65af-4bcf-8b9f-bc70b979898e-maas-api-tls\") pod \"maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms\" (UID: \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:22.264284 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:22.264208    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:22.602676 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:22.602650    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"]
Jul 16 11:02:22.604692 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:02:22.604665    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7da81d0_65af_4bcf_8b9f_bc70b979898e.slice/crio-51a5decec44dcc80de167793a08ba3ba03fd272b10294bc115f771e529542d08 WatchSource:0}: Error finding container 51a5decec44dcc80de167793a08ba3ba03fd272b10294bc115f771e529542d08: Status 404 returned error can't find the container with id 51a5decec44dcc80de167793a08ba3ba03fd272b10294bc115f771e529542d08
Jul 16 11:02:23.607920 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:23.607888    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms" event={"ID":"b7da81d0-65af-4bcf-8b9f-bc70b979898e","Type":"ContainerStarted","Data":"3ef75f4fb66d93dc081fb02c66fad9e5e96cdad7454ef9fd7f6cbee422616ec9"}
Jul 16 11:02:23.608260 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:23.607938    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms" event={"ID":"b7da81d0-65af-4bcf-8b9f-bc70b979898e","Type":"ContainerStarted","Data":"51a5decec44dcc80de167793a08ba3ba03fd272b10294bc115f771e529542d08"}
Jul 16 11:02:23.608260 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:23.608020    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:23.625351 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:23.625311    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms" podStartSLOduration=2.324233049 podStartE2EDuration="2.625298678s" totalImagesPullingTime="301.065629ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:02:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:02:23.625195913 +0000 UTC m=+1798.362231185" watchObservedRunningTime="2026-07-16 11:02:23.625298678 +0000 UTC m=+1798.362333948"
Jul 16 11:02:27.922216 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:27.922100    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 11:02:27.933400 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:27.930099    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 11:02:29.617819 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:29.617788    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:02:39.803693 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:39.803665    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"]
Jul 16 11:02:39.807206 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:39.807189    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:39.810765 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:39.810730    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5e601d\"/\"kube-root-ca.crt\""
Jul 16 11:02:39.810765 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:39.810734    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5e601d\"/\"openshift-service-ca.crt\""
Jul 16 11:02:39.810906 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:39.810734    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5e601d\"/\"test-model-5e601d-kserve-self-signed-certs\""
Jul 16 11:02:39.811170 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:39.811154    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5e601d\"/\"default-dockercfg-pkclq\""
Jul 16 11:02:39.818578 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:39.818560    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"]
Jul 16 11:02:39.948107 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:39.948081    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-home\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:39.948216 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:39.948116    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-tmp-dir\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:39.948216 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:39.948135    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-dshm\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:39.948216 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:39.948174    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4gjm\" (UniqueName: \"kubernetes.io/projected/ba5f1439-2e19-4c85-ae8f-883897c7dec8-kube-api-access-h4gjm\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:39.948317 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:39.948227    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-model-cache\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:39.948317 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:39.948257    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5f1439-2e19-4c85-ae8f-883897c7dec8-tls-certs\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.049543 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.049521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-home\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.049643 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.049554    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-tmp-dir\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.049643 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.049575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-dshm\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.049643 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.049593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4gjm\" (UniqueName: \"kubernetes.io/projected/ba5f1439-2e19-4c85-ae8f-883897c7dec8-kube-api-access-h4gjm\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.049643 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.049618    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-model-cache\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.049643 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.049640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5f1439-2e19-4c85-ae8f-883897c7dec8-tls-certs\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.049966 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.049910    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-home\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.050023 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.049973    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-model-cache\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.050023 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.050004    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-tmp-dir\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.051944 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.051923    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-dshm\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.052116 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.052100    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5f1439-2e19-4c85-ae8f-883897c7dec8-tls-certs\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.058689 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.058632    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4gjm\" (UniqueName: \"kubernetes.io/projected/ba5f1439-2e19-4c85-ae8f-883897c7dec8-kube-api-access-h4gjm\") pod \"test-model-5e601d-kserve-f89df67c9-l9t4n\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") " pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.116472 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.116451    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:40.246334 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.246206    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"]
Jul 16 11:02:40.249027 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:02:40.248995    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba5f1439_2e19_4c85_ae8f_883897c7dec8.slice/crio-3862987a4962841898750419c98b8f1ca50dafd6b70ab149f2706e760b9e5b1d WatchSource:0}: Error finding container 3862987a4962841898750419c98b8f1ca50dafd6b70ab149f2706e760b9e5b1d: Status 404 returned error can't find the container with id 3862987a4962841898750419c98b8f1ca50dafd6b70ab149f2706e760b9e5b1d
Jul 16 11:02:40.690593 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.690556    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n" event={"ID":"ba5f1439-2e19-4c85-ae8f-883897c7dec8","Type":"ContainerStarted","Data":"760154b0eb1e3438b84b9f3be901b1fbaea23cf128f5061615123e698ed77209"}
Jul 16 11:02:40.690778 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.690604    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n" event={"ID":"ba5f1439-2e19-4c85-ae8f-883897c7dec8","Type":"ContainerStarted","Data":"3862987a4962841898750419c98b8f1ca50dafd6b70ab149f2706e760b9e5b1d"}
Jul 16 11:02:40.712213 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:40.712155    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n" podStartSLOduration=1.712139697 podStartE2EDuration="1.712139697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:02:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:02:40.710662406 +0000 UTC m=+1815.447697676" watchObservedRunningTime="2026-07-16 11:02:40.712139697 +0000 UTC m=+1815.449174968"
Jul 16 11:02:50.117122 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:50.117084    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:50.117122 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:50.117131    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:50.129252 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:50.129229    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:02:50.748736 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:02:50.748704    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:03:06.399484 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.399449    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"]
Jul 16 11:03:06.402805 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.402789    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.405979 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.405958    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d156b9\"/\"default-dockercfg-xvqzl\""
Jul 16 11:03:06.406106 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.405958    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d156b9\"/\"openshift-service-ca.crt\""
Jul 16 11:03:06.406106 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.405992    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d156b9\"/\"test-model-d156b9-kserve-self-signed-certs\""
Jul 16 11:03:06.406106 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.406034    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d156b9\"/\"kube-root-ca.crt\""
Jul 16 11:03:06.412944 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.412921    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"]
Jul 16 11:03:06.482983 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.482962    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-dshm\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.483102 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.483010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-model-cache\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.483102 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.483035    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60a465e-aade-4258-81f7-366f57adc0fa-tls-certs\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.483207 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.483143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-home\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.483262 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.483226    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjf9m\" (UniqueName: \"kubernetes.io/projected/c60a465e-aade-4258-81f7-366f57adc0fa-kube-api-access-sjf9m\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.483317 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.483256    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-tmp-dir\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.583874 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.583851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-home\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.583996 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.583894    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjf9m\" (UniqueName: \"kubernetes.io/projected/c60a465e-aade-4258-81f7-366f57adc0fa-kube-api-access-sjf9m\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.583996 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.583914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-tmp-dir\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.583996 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.583950    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-dshm\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.583996 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.583974    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-model-cache\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.583996 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.583989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60a465e-aade-4258-81f7-366f57adc0fa-tls-certs\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.584325 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.584295    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-home\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.584416 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.584368    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-tmp-dir\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.584465 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.584408    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-model-cache\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.586056 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.586028    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-dshm\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.586442 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.586423    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60a465e-aade-4258-81f7-366f57adc0fa-tls-certs\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.593080 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.593061    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjf9m\" (UniqueName: \"kubernetes.io/projected/c60a465e-aade-4258-81f7-366f57adc0fa-kube-api-access-sjf9m\") pod \"test-model-d156b9-kserve-69c87d5899-jggzg\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") " pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.713302 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.713249    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:06.849063 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:06.849037    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"]
Jul 16 11:03:06.850330 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:03:06.850301    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc60a465e_aade_4258_81f7_366f57adc0fa.slice/crio-2dcae8f8c94da954b58e18d20247b2dc4c10f6c08c9f077e2d9093b56403b89e WatchSource:0}: Error finding container 2dcae8f8c94da954b58e18d20247b2dc4c10f6c08c9f077e2d9093b56403b89e: Status 404 returned error can't find the container with id 2dcae8f8c94da954b58e18d20247b2dc4c10f6c08c9f077e2d9093b56403b89e
Jul 16 11:03:07.813339 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:07.813300    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg" event={"ID":"c60a465e-aade-4258-81f7-366f57adc0fa","Type":"ContainerStarted","Data":"fc625df8a556ac1505d83450edb0da2311815c769e9b36d90bf5584e12af042d"}
Jul 16 11:03:07.813339 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:07.813345    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg" event={"ID":"c60a465e-aade-4258-81f7-366f57adc0fa","Type":"ContainerStarted","Data":"2dcae8f8c94da954b58e18d20247b2dc4c10f6c08c9f077e2d9093b56403b89e"}
Jul 16 11:03:07.835296 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:07.835246    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg" podStartSLOduration=1.835233071 podStartE2EDuration="1.835233071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:03:07.83348252 +0000 UTC m=+1842.570517806" watchObservedRunningTime="2026-07-16 11:03:07.835233071 +0000 UTC m=+1842.572268382"
Jul 16 11:03:16.714069 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:16.714032    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:16.714634 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:16.714091    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:16.726188 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:16.726164    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:16.865364 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:16.865338    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:41.827623 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:41.827592    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"]
Jul 16 11:03:42.838198 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:42.838165    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"]
Jul 16 11:03:56.847453 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:56.847379    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n" podUID="ba5f1439-2e19-4c85-ae8f-883897c7dec8" containerName="main" containerID="cri-o://760154b0eb1e3438b84b9f3be901b1fbaea23cf128f5061615123e698ed77209" gracePeriod=45
Jul 16 11:03:57.033570 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.033524    2576 generic.go:356] "Generic (PLEG): container finished" podID="ba5f1439-2e19-4c85-ae8f-883897c7dec8" containerID="760154b0eb1e3438b84b9f3be901b1fbaea23cf128f5061615123e698ed77209" exitCode=0
Jul 16 11:03:57.033761 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.033580    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n" event={"ID":"ba5f1439-2e19-4c85-ae8f-883897c7dec8","Type":"ContainerDied","Data":"760154b0eb1e3438b84b9f3be901b1fbaea23cf128f5061615123e698ed77209"}
Jul 16 11:03:57.092581 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.092550    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:03:57.213057 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.213024    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4gjm\" (UniqueName: \"kubernetes.io/projected/ba5f1439-2e19-4c85-ae8f-883897c7dec8-kube-api-access-h4gjm\") pod \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") "
Jul 16 11:03:57.213223 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.213081    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-model-cache\") pod \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") "
Jul 16 11:03:57.213223 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.213127    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-dshm\") pod \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") "
Jul 16 11:03:57.213223 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.213157    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-home\") pod \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") "
Jul 16 11:03:57.213223 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.213200    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5f1439-2e19-4c85-ae8f-883897c7dec8-tls-certs\") pod \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") "
Jul 16 11:03:57.213441 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.213240    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-tmp-dir\") pod \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\" (UID: \"ba5f1439-2e19-4c85-ae8f-883897c7dec8\") "
Jul 16 11:03:57.213441 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.213368    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-model-cache" (OuterVolumeSpecName: "model-cache") pod "ba5f1439-2e19-4c85-ae8f-883897c7dec8" (UID: "ba5f1439-2e19-4c85-ae8f-883897c7dec8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:03:57.213535 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.213473    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-home" (OuterVolumeSpecName: "home") pod "ba5f1439-2e19-4c85-ae8f-883897c7dec8" (UID: "ba5f1439-2e19-4c85-ae8f-883897c7dec8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:03:57.213613 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.213595    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-home\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:03:57.213668 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.213621    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-model-cache\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:03:57.213705 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.213665    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ba5f1439-2e19-4c85-ae8f-883897c7dec8" (UID: "ba5f1439-2e19-4c85-ae8f-883897c7dec8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:03:57.215368 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.215342    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba5f1439-2e19-4c85-ae8f-883897c7dec8-kube-api-access-h4gjm" (OuterVolumeSpecName: "kube-api-access-h4gjm") pod "ba5f1439-2e19-4c85-ae8f-883897c7dec8" (UID: "ba5f1439-2e19-4c85-ae8f-883897c7dec8"). InnerVolumeSpecName "kube-api-access-h4gjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:03:57.215616 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.215595    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba5f1439-2e19-4c85-ae8f-883897c7dec8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ba5f1439-2e19-4c85-ae8f-883897c7dec8" (UID: "ba5f1439-2e19-4c85-ae8f-883897c7dec8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:03:57.215677 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.215600    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-dshm" (OuterVolumeSpecName: "dshm") pod "ba5f1439-2e19-4c85-ae8f-883897c7dec8" (UID: "ba5f1439-2e19-4c85-ae8f-883897c7dec8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:03:57.314227 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.314193    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4gjm\" (UniqueName: \"kubernetes.io/projected/ba5f1439-2e19-4c85-ae8f-883897c7dec8-kube-api-access-h4gjm\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:03:57.314227 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.314221    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-dshm\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:03:57.314227 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.314234    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5f1439-2e19-4c85-ae8f-883897c7dec8-tls-certs\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:03:57.314446 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.314242    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5f1439-2e19-4c85-ae8f-883897c7dec8-tmp-dir\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:03:57.859144 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:57.859106    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg" podUID="c60a465e-aade-4258-81f7-366f57adc0fa" containerName="main" containerID="cri-o://fc625df8a556ac1505d83450edb0da2311815c769e9b36d90bf5584e12af042d" gracePeriod=45
Jul 16 11:03:58.051555 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.051455    2576 generic.go:356] "Generic (PLEG): container finished" podID="c60a465e-aade-4258-81f7-366f57adc0fa" containerID="fc625df8a556ac1505d83450edb0da2311815c769e9b36d90bf5584e12af042d" exitCode=0
Jul 16 11:03:58.051555 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.051498    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg" event={"ID":"c60a465e-aade-4258-81f7-366f57adc0fa","Type":"ContainerDied","Data":"fc625df8a556ac1505d83450edb0da2311815c769e9b36d90bf5584e12af042d"}
Jul 16 11:03:58.053050 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.053029    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n" event={"ID":"ba5f1439-2e19-4c85-ae8f-883897c7dec8","Type":"ContainerDied","Data":"3862987a4962841898750419c98b8f1ca50dafd6b70ab149f2706e760b9e5b1d"}
Jul 16 11:03:58.053050 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.053054    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"
Jul 16 11:03:58.053226 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.053064    2576 scope.go:117] "RemoveContainer" containerID="760154b0eb1e3438b84b9f3be901b1fbaea23cf128f5061615123e698ed77209"
Jul 16 11:03:58.071080 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.071056    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"]
Jul 16 11:03:58.075766 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.075725    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-5e601d/test-model-5e601d-kserve-f89df67c9-l9t4n"]
Jul 16 11:03:58.113725 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.113708    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:58.219584 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.219550    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-home\") pod \"c60a465e-aade-4258-81f7-366f57adc0fa\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") "
Jul 16 11:03:58.219772 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.219594    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60a465e-aade-4258-81f7-366f57adc0fa-tls-certs\") pod \"c60a465e-aade-4258-81f7-366f57adc0fa\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") "
Jul 16 11:03:58.219772 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.219638    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-tmp-dir\") pod \"c60a465e-aade-4258-81f7-366f57adc0fa\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") "
Jul 16 11:03:58.219772 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.219670    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-dshm\") pod \"c60a465e-aade-4258-81f7-366f57adc0fa\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") "
Jul 16 11:03:58.219772 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.219714    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-model-cache\") pod \"c60a465e-aade-4258-81f7-366f57adc0fa\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") "
Jul 16 11:03:58.219993 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.219841    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjf9m\" (UniqueName: \"kubernetes.io/projected/c60a465e-aade-4258-81f7-366f57adc0fa-kube-api-access-sjf9m\") pod \"c60a465e-aade-4258-81f7-366f57adc0fa\" (UID: \"c60a465e-aade-4258-81f7-366f57adc0fa\") "
Jul 16 11:03:58.219993 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.219844    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-home" (OuterVolumeSpecName: "home") pod "c60a465e-aade-4258-81f7-366f57adc0fa" (UID: "c60a465e-aade-4258-81f7-366f57adc0fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:03:58.219993 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.219945    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c60a465e-aade-4258-81f7-366f57adc0fa" (UID: "c60a465e-aade-4258-81f7-366f57adc0fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:03:58.220113 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.220027    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "c60a465e-aade-4258-81f7-366f57adc0fa" (UID: "c60a465e-aade-4258-81f7-366f57adc0fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:03:58.220156 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.220133    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-home\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:03:58.220198 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.220167    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-tmp-dir\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:03:58.220198 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.220178    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-model-cache\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:03:58.221972 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.221950    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c60a465e-aade-4258-81f7-366f57adc0fa-kube-api-access-sjf9m" (OuterVolumeSpecName: "kube-api-access-sjf9m") pod "c60a465e-aade-4258-81f7-366f57adc0fa" (UID: "c60a465e-aade-4258-81f7-366f57adc0fa"). InnerVolumeSpecName "kube-api-access-sjf9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:03:58.222221 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.222195    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-dshm" (OuterVolumeSpecName: "dshm") pod "c60a465e-aade-4258-81f7-366f57adc0fa" (UID: "c60a465e-aade-4258-81f7-366f57adc0fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:03:58.222221 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.222203    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c60a465e-aade-4258-81f7-366f57adc0fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c60a465e-aade-4258-81f7-366f57adc0fa" (UID: "c60a465e-aade-4258-81f7-366f57adc0fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:03:58.321147 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.321093    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjf9m\" (UniqueName: \"kubernetes.io/projected/c60a465e-aade-4258-81f7-366f57adc0fa-kube-api-access-sjf9m\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:03:58.321147 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.321113    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60a465e-aade-4258-81f7-366f57adc0fa-tls-certs\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:03:58.321147 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:58.321123    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60a465e-aade-4258-81f7-366f57adc0fa-dshm\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:03:59.059216 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:59.059180    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg" event={"ID":"c60a465e-aade-4258-81f7-366f57adc0fa","Type":"ContainerDied","Data":"2dcae8f8c94da954b58e18d20247b2dc4c10f6c08c9f077e2d9093b56403b89e"}
Jul 16 11:03:59.059216 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:59.059214    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"
Jul 16 11:03:59.059657 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:59.059235    2576 scope.go:117] "RemoveContainer" containerID="fc625df8a556ac1505d83450edb0da2311815c769e9b36d90bf5584e12af042d"
Jul 16 11:03:59.080779 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:59.080729    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"]
Jul 16 11:03:59.087539 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:59.087517    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-d156b9/test-model-d156b9-kserve-69c87d5899-jggzg"]
Jul 16 11:03:59.801390 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:59.801357    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba5f1439-2e19-4c85-ae8f-883897c7dec8" path="/var/lib/kubelet/pods/ba5f1439-2e19-4c85-ae8f-883897c7dec8/volumes"
Jul 16 11:03:59.801763 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:03:59.801732    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c60a465e-aade-4258-81f7-366f57adc0fa" path="/var/lib/kubelet/pods/c60a465e-aade-4258-81f7-366f57adc0fa/volumes"
Jul 16 11:04:24.941881 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:24.941847    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"]
Jul 16 11:04:24.942275 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:24.942052    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99" podUID="f1803d69-370b-4b11-bc3e-511e7a9fbc13" containerName="maas-api" containerID="cri-o://848f7ba9a4ad319aa2af888db14c037d46a2cdbeea880d728517d6e5563038e6" gracePeriod=30
Jul 16 11:04:25.180532 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.180499    2576 generic.go:356] "Generic (PLEG): container finished" podID="f1803d69-370b-4b11-bc3e-511e7a9fbc13" containerID="848f7ba9a4ad319aa2af888db14c037d46a2cdbeea880d728517d6e5563038e6" exitCode=0
Jul 16 11:04:25.180682 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.180571    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99" event={"ID":"f1803d69-370b-4b11-bc3e-511e7a9fbc13","Type":"ContainerDied","Data":"848f7ba9a4ad319aa2af888db14c037d46a2cdbeea880d728517d6e5563038e6"}
Jul 16 11:04:25.180682 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.180643    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99" event={"ID":"f1803d69-370b-4b11-bc3e-511e7a9fbc13","Type":"ContainerDied","Data":"6249bfb6ed4855e4dded8722aed2403727163800cee9cd65457026c381b5e738"}
Jul 16 11:04:25.180682 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.180659    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6249bfb6ed4855e4dded8722aed2403727163800cee9cd65457026c381b5e738"
Jul 16 11:04:25.187974 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.187953    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:04:25.344441 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.344364    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvhcq\" (UniqueName: \"kubernetes.io/projected/f1803d69-370b-4b11-bc3e-511e7a9fbc13-kube-api-access-hvhcq\") pod \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\" (UID: \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\") "
Jul 16 11:04:25.344441 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.344433    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1803d69-370b-4b11-bc3e-511e7a9fbc13-maas-api-tls\") pod \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\" (UID: \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\") "
Jul 16 11:04:25.344652 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.344523    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1803d69-370b-4b11-bc3e-511e7a9fbc13-openshift-service-ca\") pod \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\" (UID: \"f1803d69-370b-4b11-bc3e-511e7a9fbc13\") "
Jul 16 11:04:25.344898 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.344872    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1803d69-370b-4b11-bc3e-511e7a9fbc13-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f1803d69-370b-4b11-bc3e-511e7a9fbc13" (UID: "f1803d69-370b-4b11-bc3e-511e7a9fbc13"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:04:25.346517 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.346491    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1803d69-370b-4b11-bc3e-511e7a9fbc13-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f1803d69-370b-4b11-bc3e-511e7a9fbc13" (UID: "f1803d69-370b-4b11-bc3e-511e7a9fbc13"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:04:25.346623 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.346524    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1803d69-370b-4b11-bc3e-511e7a9fbc13-kube-api-access-hvhcq" (OuterVolumeSpecName: "kube-api-access-hvhcq") pod "f1803d69-370b-4b11-bc3e-511e7a9fbc13" (UID: "f1803d69-370b-4b11-bc3e-511e7a9fbc13"). InnerVolumeSpecName "kube-api-access-hvhcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:04:25.445634 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.445606    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1803d69-370b-4b11-bc3e-511e7a9fbc13-maas-api-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:04:25.445634 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.445633    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1803d69-370b-4b11-bc3e-511e7a9fbc13-openshift-service-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:04:25.445790 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:25.445649    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvhcq\" (UniqueName: \"kubernetes.io/projected/f1803d69-370b-4b11-bc3e-511e7a9fbc13-kube-api-access-hvhcq\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:04:26.184406 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:26.184380    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"
Jul 16 11:04:26.202408 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:26.202382    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"]
Jul 16 11:04:26.206299 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:26.206279    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5e601d-6d7cc65788-nsw99"]
Jul 16 11:04:27.801155 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:27.801124    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1803d69-370b-4b11-bc3e-511e7a9fbc13" path="/var/lib/kubelet/pods/f1803d69-370b-4b11-bc3e-511e7a9fbc13/volumes"
Jul 16 11:04:40.031307 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.031272    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"]
Jul 16 11:04:40.031851 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.031463    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms" podUID="b7da81d0-65af-4bcf-8b9f-bc70b979898e" containerName="maas-api" containerID="cri-o://3ef75f4fb66d93dc081fb02c66fad9e5e96cdad7454ef9fd7f6cbee422616ec9" gracePeriod=30
Jul 16 11:04:40.246324 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.246285    2576 generic.go:356] "Generic (PLEG): container finished" podID="b7da81d0-65af-4bcf-8b9f-bc70b979898e" containerID="3ef75f4fb66d93dc081fb02c66fad9e5e96cdad7454ef9fd7f6cbee422616ec9" exitCode=0
Jul 16 11:04:40.246463 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.246350    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms" event={"ID":"b7da81d0-65af-4bcf-8b9f-bc70b979898e","Type":"ContainerDied","Data":"3ef75f4fb66d93dc081fb02c66fad9e5e96cdad7454ef9fd7f6cbee422616ec9"}
Jul 16 11:04:40.277433 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.277415    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:04:40.364738 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.364718    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7da81d0-65af-4bcf-8b9f-bc70b979898e-openshift-service-ca\") pod \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\" (UID: \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\") "
Jul 16 11:04:40.364868 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.364788    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7da81d0-65af-4bcf-8b9f-bc70b979898e-maas-api-tls\") pod \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\" (UID: \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\") "
Jul 16 11:04:40.364868 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.364817    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7vfn\" (UniqueName: \"kubernetes.io/projected/b7da81d0-65af-4bcf-8b9f-bc70b979898e-kube-api-access-j7vfn\") pod \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\" (UID: \"b7da81d0-65af-4bcf-8b9f-bc70b979898e\") "
Jul 16 11:04:40.365097 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.365070    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7da81d0-65af-4bcf-8b9f-bc70b979898e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b7da81d0-65af-4bcf-8b9f-bc70b979898e" (UID: "b7da81d0-65af-4bcf-8b9f-bc70b979898e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:04:40.366665 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.366637    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7da81d0-65af-4bcf-8b9f-bc70b979898e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b7da81d0-65af-4bcf-8b9f-bc70b979898e" (UID: "b7da81d0-65af-4bcf-8b9f-bc70b979898e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:04:40.366788 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.366710    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7da81d0-65af-4bcf-8b9f-bc70b979898e-kube-api-access-j7vfn" (OuterVolumeSpecName: "kube-api-access-j7vfn") pod "b7da81d0-65af-4bcf-8b9f-bc70b979898e" (UID: "b7da81d0-65af-4bcf-8b9f-bc70b979898e"). InnerVolumeSpecName "kube-api-access-j7vfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:04:40.465998 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.465973    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7da81d0-65af-4bcf-8b9f-bc70b979898e-maas-api-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:04:40.465998 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.465995    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7vfn\" (UniqueName: \"kubernetes.io/projected/b7da81d0-65af-4bcf-8b9f-bc70b979898e-kube-api-access-j7vfn\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:04:40.466122 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:40.466005    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7da81d0-65af-4bcf-8b9f-bc70b979898e-openshift-service-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:04:41.252046 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:41.252019    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"
Jul 16 11:04:41.252046 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:41.252024    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms" event={"ID":"b7da81d0-65af-4bcf-8b9f-bc70b979898e","Type":"ContainerDied","Data":"51a5decec44dcc80de167793a08ba3ba03fd272b10294bc115f771e529542d08"}
Jul 16 11:04:41.252477 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:41.252080    2576 scope.go:117] "RemoveContainer" containerID="3ef75f4fb66d93dc081fb02c66fad9e5e96cdad7454ef9fd7f6cbee422616ec9"
Jul 16 11:04:41.284116 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:41.284093    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"]
Jul 16 11:04:41.291189 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:41.291166    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d156b9-89c87646c-5s8ms"]
Jul 16 11:04:41.800339 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:41.800308    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7da81d0-65af-4bcf-8b9f-bc70b979898e" path="/var/lib/kubelet/pods/b7da81d0-65af-4bcf-8b9f-bc70b979898e/volumes"
Jul 16 11:04:49.634945 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.634914    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"]
Jul 16 11:04:49.635321 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.635261    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1803d69-370b-4b11-bc3e-511e7a9fbc13" containerName="maas-api"
Jul 16 11:04:49.635321 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.635271    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1803d69-370b-4b11-bc3e-511e7a9fbc13" containerName="maas-api"
Jul 16 11:04:49.635321 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.635283    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7da81d0-65af-4bcf-8b9f-bc70b979898e" containerName="maas-api"
Jul 16 11:04:49.635321 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.635288    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7da81d0-65af-4bcf-8b9f-bc70b979898e" containerName="maas-api"
Jul 16 11:04:49.635321 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.635301    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c60a465e-aade-4258-81f7-366f57adc0fa" containerName="main"
Jul 16 11:04:49.635321 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.635306    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c60a465e-aade-4258-81f7-366f57adc0fa" containerName="main"
Jul 16 11:04:49.635321 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.635313    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba5f1439-2e19-4c85-ae8f-883897c7dec8" containerName="main"
Jul 16 11:04:49.635321 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.635318    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba5f1439-2e19-4c85-ae8f-883897c7dec8" containerName="main"
Jul 16 11:04:49.635601 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.635371    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1803d69-370b-4b11-bc3e-511e7a9fbc13" containerName="maas-api"
Jul 16 11:04:49.635601 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.635380    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba5f1439-2e19-4c85-ae8f-883897c7dec8" containerName="main"
Jul 16 11:04:49.635601 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.635386    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c60a465e-aade-4258-81f7-366f57adc0fa" containerName="main"
Jul 16 11:04:49.635601 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.635392    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7da81d0-65af-4bcf-8b9f-bc70b979898e" containerName="maas-api"
Jul 16 11:04:49.639588 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.639572    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:04:49.642317 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.642290    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 11:04:49.642918 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.642898    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 11:04:49.643017 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.642904    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-270141-serving-cert\""
Jul 16 11:04:49.643172 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.643156    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kn7cc\""
Jul 16 11:04:49.649927 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.649905    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"]
Jul 16 11:04:49.741033 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.741002    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56822c92-e464-46d9-873d-d5523722d800-maas-api-tls\") pod \"maas-api-e2e-api-a-270141-8fddd85cf-wwjrb\" (UID: \"56822c92-e464-46d9-873d-d5523722d800\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:04:49.741172 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.741040    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56822c92-e464-46d9-873d-d5523722d800-openshift-service-ca\") pod \"maas-api-e2e-api-a-270141-8fddd85cf-wwjrb\" (UID: \"56822c92-e464-46d9-873d-d5523722d800\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:04:49.741172 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.741085    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc54j\" (UniqueName: \"kubernetes.io/projected/56822c92-e464-46d9-873d-d5523722d800-kube-api-access-cc54j\") pod \"maas-api-e2e-api-a-270141-8fddd85cf-wwjrb\" (UID: \"56822c92-e464-46d9-873d-d5523722d800\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:04:49.841791 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.841739    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc54j\" (UniqueName: \"kubernetes.io/projected/56822c92-e464-46d9-873d-d5523722d800-kube-api-access-cc54j\") pod \"maas-api-e2e-api-a-270141-8fddd85cf-wwjrb\" (UID: \"56822c92-e464-46d9-873d-d5523722d800\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:04:49.841922 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.841843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56822c92-e464-46d9-873d-d5523722d800-maas-api-tls\") pod \"maas-api-e2e-api-a-270141-8fddd85cf-wwjrb\" (UID: \"56822c92-e464-46d9-873d-d5523722d800\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:04:49.841922 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.841865    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56822c92-e464-46d9-873d-d5523722d800-openshift-service-ca\") pod \"maas-api-e2e-api-a-270141-8fddd85cf-wwjrb\" (UID: \"56822c92-e464-46d9-873d-d5523722d800\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:04:49.842402 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.842385    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56822c92-e464-46d9-873d-d5523722d800-openshift-service-ca\") pod \"maas-api-e2e-api-a-270141-8fddd85cf-wwjrb\" (UID: \"56822c92-e464-46d9-873d-d5523722d800\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:04:49.844122 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.844101    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56822c92-e464-46d9-873d-d5523722d800-maas-api-tls\") pod \"maas-api-e2e-api-a-270141-8fddd85cf-wwjrb\" (UID: \"56822c92-e464-46d9-873d-d5523722d800\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:04:49.854866 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.854843    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc54j\" (UniqueName: \"kubernetes.io/projected/56822c92-e464-46d9-873d-d5523722d800-kube-api-access-cc54j\") pod \"maas-api-e2e-api-a-270141-8fddd85cf-wwjrb\" (UID: \"56822c92-e464-46d9-873d-d5523722d800\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:04:49.950911 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:49.950850    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:04:50.090608 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:50.090585    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"]
Jul 16 11:04:50.092275 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:04:50.092252    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56822c92_e464_46d9_873d_d5523722d800.slice/crio-77d003b6cc55eec4cf0ebedaa4fd27ad90ef511efc5cd441bfedf2b592eb590b WatchSource:0}: Error finding container 77d003b6cc55eec4cf0ebedaa4fd27ad90ef511efc5cd441bfedf2b592eb590b: Status 404 returned error can't find the container with id 77d003b6cc55eec4cf0ebedaa4fd27ad90ef511efc5cd441bfedf2b592eb590b
Jul 16 11:04:50.094037 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:50.094019    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 11:04:50.293780 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:50.293687    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb" event={"ID":"56822c92-e464-46d9-873d-d5523722d800","Type":"ContainerStarted","Data":"77d003b6cc55eec4cf0ebedaa4fd27ad90ef511efc5cd441bfedf2b592eb590b"}
Jul 16 11:04:51.299235 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:51.299200    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb" event={"ID":"56822c92-e464-46d9-873d-d5523722d800","Type":"ContainerStarted","Data":"7af6b7548531a23e899cdfdaceb8bff37bdc3ea619b66867c1ebb7eaf5c8f891"}
Jul 16 11:04:51.299629 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:51.299323    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:04:51.316041 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:51.315989    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb" podStartSLOduration=1.844467758 podStartE2EDuration="2.315972238s" totalImagesPullingTime="471.50448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:04:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:51.315301552 +0000 UTC m=+1946.052336824" watchObservedRunningTime="2026-07-16 11:04:51.315972238 +0000 UTC m=+1946.053007510"
Jul 16 11:04:57.308761 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:57.308720    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:04:58.197240 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.197207    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"]
Jul 16 11:04:58.201119 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.201097    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:04:58.204048 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.204028    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-89289a-serving-cert\""
Jul 16 11:04:58.213434 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.213413    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"]
Jul 16 11:04:58.312679 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.312656    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-maas-api-tls\") pod \"maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2\" (UID: \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:04:58.313013 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.312693    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rltpd\" (UniqueName: \"kubernetes.io/projected/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-kube-api-access-rltpd\") pod \"maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2\" (UID: \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:04:58.313013 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.312724    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-openshift-service-ca\") pod \"maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2\" (UID: \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:04:58.413091 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.413064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-maas-api-tls\") pod \"maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2\" (UID: \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:04:58.413218 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.413109    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rltpd\" (UniqueName: \"kubernetes.io/projected/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-kube-api-access-rltpd\") pod \"maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2\" (UID: \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:04:58.413218 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.413150    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-openshift-service-ca\") pod \"maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2\" (UID: \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:04:58.413218 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:04:58.413204    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-serving-cert: secret "maas-api-e2e-api-b-89289a-serving-cert" not found
Jul 16 11:04:58.413384 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:04:58.413265    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-maas-api-tls podName:7c722a34-6c8b-44f5-bbfa-7e3cc07d7514 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:58.913249866 +0000 UTC m=+1953.650285114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-maas-api-tls") pod "maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2" (UID: "7c722a34-6c8b-44f5-bbfa-7e3cc07d7514") : secret "maas-api-e2e-api-b-89289a-serving-cert" not found
Jul 16 11:04:58.413801 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.413782    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-openshift-service-ca\") pod \"maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2\" (UID: \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:04:58.427990 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.427973    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rltpd\" (UniqueName: \"kubernetes.io/projected/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-kube-api-access-rltpd\") pod \"maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2\" (UID: \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:04:58.917062 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.917026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-maas-api-tls\") pod \"maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2\" (UID: \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:04:58.919228 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:58.919203    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-maas-api-tls\") pod \"maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2\" (UID: \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:04:59.115072 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:59.115035    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:04:59.249958 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:59.249082    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"]
Jul 16 11:04:59.335925 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:04:59.335892    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2" event={"ID":"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514","Type":"ContainerStarted","Data":"fc270a3185fe7a2e0a2b22eac1708398b762b1ced5340840b81df0ce3203581c"}
Jul 16 11:05:00.341873 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:00.341833    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2" event={"ID":"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514","Type":"ContainerStarted","Data":"832e0a2c4c3c65c890d113c182bcaa150fe928708e419e254d56108c1df4bee5"}
Jul 16 11:05:00.342263 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:00.341926    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:05:00.367957 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:00.367906    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2" podStartSLOduration=2.055315388 podStartE2EDuration="2.367888016s" totalImagesPullingTime="312.572628ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:04:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:05:00.367846861 +0000 UTC m=+1955.104882136" watchObservedRunningTime="2026-07-16 11:05:00.367888016 +0000 UTC m=+1955.104923300"
Jul 16 11:05:06.351622 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:06.351592    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:05:17.741098 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:17.741021    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"]
Jul 16 11:05:17.741501 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:17.741293    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb" podUID="56822c92-e464-46d9-873d-d5523722d800" containerName="maas-api" containerID="cri-o://7af6b7548531a23e899cdfdaceb8bff37bdc3ea619b66867c1ebb7eaf5c8f891" gracePeriod=30
Jul 16 11:05:17.987532 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:17.987508    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:05:18.073809 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.073728    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56822c92-e464-46d9-873d-d5523722d800-maas-api-tls\") pod \"56822c92-e464-46d9-873d-d5523722d800\" (UID: \"56822c92-e464-46d9-873d-d5523722d800\") "
Jul 16 11:05:18.073809 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.073778    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc54j\" (UniqueName: \"kubernetes.io/projected/56822c92-e464-46d9-873d-d5523722d800-kube-api-access-cc54j\") pod \"56822c92-e464-46d9-873d-d5523722d800\" (UID: \"56822c92-e464-46d9-873d-d5523722d800\") "
Jul 16 11:05:18.074011 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.073878    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56822c92-e464-46d9-873d-d5523722d800-openshift-service-ca\") pod \"56822c92-e464-46d9-873d-d5523722d800\" (UID: \"56822c92-e464-46d9-873d-d5523722d800\") "
Jul 16 11:05:18.074154 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.074128    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56822c92-e464-46d9-873d-d5523722d800-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "56822c92-e464-46d9-873d-d5523722d800" (UID: "56822c92-e464-46d9-873d-d5523722d800"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:05:18.075810 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.075786    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56822c92-e464-46d9-873d-d5523722d800-kube-api-access-cc54j" (OuterVolumeSpecName: "kube-api-access-cc54j") pod "56822c92-e464-46d9-873d-d5523722d800" (UID: "56822c92-e464-46d9-873d-d5523722d800"). InnerVolumeSpecName "kube-api-access-cc54j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:05:18.075940 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.075879    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56822c92-e464-46d9-873d-d5523722d800-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "56822c92-e464-46d9-873d-d5523722d800" (UID: "56822c92-e464-46d9-873d-d5523722d800"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:05:18.175519 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.175490    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56822c92-e464-46d9-873d-d5523722d800-openshift-service-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:05:18.175519 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.175517    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56822c92-e464-46d9-873d-d5523722d800-maas-api-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:05:18.175661 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.175531    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc54j\" (UniqueName: \"kubernetes.io/projected/56822c92-e464-46d9-873d-d5523722d800-kube-api-access-cc54j\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:05:18.425444 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.425415    2576 generic.go:356] "Generic (PLEG): container finished" podID="56822c92-e464-46d9-873d-d5523722d800" containerID="7af6b7548531a23e899cdfdaceb8bff37bdc3ea619b66867c1ebb7eaf5c8f891" exitCode=0
Jul 16 11:05:18.425579 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.425458    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"
Jul 16 11:05:18.425579 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.425500    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb" event={"ID":"56822c92-e464-46d9-873d-d5523722d800","Type":"ContainerDied","Data":"7af6b7548531a23e899cdfdaceb8bff37bdc3ea619b66867c1ebb7eaf5c8f891"}
Jul 16 11:05:18.425579 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.425549    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb" event={"ID":"56822c92-e464-46d9-873d-d5523722d800","Type":"ContainerDied","Data":"77d003b6cc55eec4cf0ebedaa4fd27ad90ef511efc5cd441bfedf2b592eb590b"}
Jul 16 11:05:18.425579 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.425573    2576 scope.go:117] "RemoveContainer" containerID="7af6b7548531a23e899cdfdaceb8bff37bdc3ea619b66867c1ebb7eaf5c8f891"
Jul 16 11:05:18.436308 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.436287    2576 scope.go:117] "RemoveContainer" containerID="7af6b7548531a23e899cdfdaceb8bff37bdc3ea619b66867c1ebb7eaf5c8f891"
Jul 16 11:05:18.436566 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:05:18.436545    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7af6b7548531a23e899cdfdaceb8bff37bdc3ea619b66867c1ebb7eaf5c8f891\": container with ID starting with 7af6b7548531a23e899cdfdaceb8bff37bdc3ea619b66867c1ebb7eaf5c8f891 not found: ID does not exist" containerID="7af6b7548531a23e899cdfdaceb8bff37bdc3ea619b66867c1ebb7eaf5c8f891"
Jul 16 11:05:18.436622 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.436574    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7af6b7548531a23e899cdfdaceb8bff37bdc3ea619b66867c1ebb7eaf5c8f891"} err="failed to get container status \"7af6b7548531a23e899cdfdaceb8bff37bdc3ea619b66867c1ebb7eaf5c8f891\": rpc error: code = NotFound desc = could not find container \"7af6b7548531a23e899cdfdaceb8bff37bdc3ea619b66867c1ebb7eaf5c8f891\": container with ID starting with 7af6b7548531a23e899cdfdaceb8bff37bdc3ea619b66867c1ebb7eaf5c8f891 not found: ID does not exist"
Jul 16 11:05:18.474265 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.474245    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"]
Jul 16 11:05:18.482824 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:18.482800    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-270141-8fddd85cf-wwjrb"]
Jul 16 11:05:19.801804 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:19.801776    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56822c92-e464-46d9-873d-d5523722d800" path="/var/lib/kubelet/pods/56822c92-e464-46d9-873d-d5523722d800/volumes"
Jul 16 11:05:32.475898 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:32.475862    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"]
Jul 16 11:05:32.476312 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:32.476065    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2" podUID="7c722a34-6c8b-44f5-bbfa-7e3cc07d7514" containerName="maas-api" containerID="cri-o://832e0a2c4c3c65c890d113c182bcaa150fe928708e419e254d56108c1df4bee5" gracePeriod=30
Jul 16 11:05:32.724004 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:32.723981    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:05:32.895538 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:32.895508    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-openshift-service-ca\") pod \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\" (UID: \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\") "
Jul 16 11:05:32.895727 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:32.895566    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rltpd\" (UniqueName: \"kubernetes.io/projected/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-kube-api-access-rltpd\") pod \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\" (UID: \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\") "
Jul 16 11:05:32.895727 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:32.895653    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-maas-api-tls\") pod \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\" (UID: \"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514\") "
Jul 16 11:05:32.895971 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:32.895940    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c722a34-6c8b-44f5-bbfa-7e3cc07d7514" (UID: "7c722a34-6c8b-44f5-bbfa-7e3cc07d7514"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:05:32.896688 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:32.896659    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-openshift-service-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:05:32.897985 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:32.897961    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c722a34-6c8b-44f5-bbfa-7e3cc07d7514" (UID: "7c722a34-6c8b-44f5-bbfa-7e3cc07d7514"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:05:32.898062 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:32.898047    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-kube-api-access-rltpd" (OuterVolumeSpecName: "kube-api-access-rltpd") pod "7c722a34-6c8b-44f5-bbfa-7e3cc07d7514" (UID: "7c722a34-6c8b-44f5-bbfa-7e3cc07d7514"). InnerVolumeSpecName "kube-api-access-rltpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:05:32.997497 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:32.997468    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-maas-api-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:05:32.997497 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:32.997493    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rltpd\" (UniqueName: \"kubernetes.io/projected/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514-kube-api-access-rltpd\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:05:33.495392 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:33.495357    2576 generic.go:356] "Generic (PLEG): container finished" podID="7c722a34-6c8b-44f5-bbfa-7e3cc07d7514" containerID="832e0a2c4c3c65c890d113c182bcaa150fe928708e419e254d56108c1df4bee5" exitCode=0
Jul 16 11:05:33.495855 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:33.495404    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"
Jul 16 11:05:33.495855 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:33.495417    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2" event={"ID":"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514","Type":"ContainerDied","Data":"832e0a2c4c3c65c890d113c182bcaa150fe928708e419e254d56108c1df4bee5"}
Jul 16 11:05:33.495855 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:33.495471    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2" event={"ID":"7c722a34-6c8b-44f5-bbfa-7e3cc07d7514","Type":"ContainerDied","Data":"fc270a3185fe7a2e0a2b22eac1708398b762b1ced5340840b81df0ce3203581c"}
Jul 16 11:05:33.495855 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:33.495501    2576 scope.go:117] "RemoveContainer" containerID="832e0a2c4c3c65c890d113c182bcaa150fe928708e419e254d56108c1df4bee5"
Jul 16 11:05:33.504783 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:33.504765    2576 scope.go:117] "RemoveContainer" containerID="832e0a2c4c3c65c890d113c182bcaa150fe928708e419e254d56108c1df4bee5"
Jul 16 11:05:33.505014 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:05:33.504995    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"832e0a2c4c3c65c890d113c182bcaa150fe928708e419e254d56108c1df4bee5\": container with ID starting with 832e0a2c4c3c65c890d113c182bcaa150fe928708e419e254d56108c1df4bee5 not found: ID does not exist" containerID="832e0a2c4c3c65c890d113c182bcaa150fe928708e419e254d56108c1df4bee5"
Jul 16 11:05:33.505070 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:33.505022    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"832e0a2c4c3c65c890d113c182bcaa150fe928708e419e254d56108c1df4bee5"} err="failed to get container status \"832e0a2c4c3c65c890d113c182bcaa150fe928708e419e254d56108c1df4bee5\": rpc error: code = NotFound desc = could not find container \"832e0a2c4c3c65c890d113c182bcaa150fe928708e419e254d56108c1df4bee5\": container with ID starting with 832e0a2c4c3c65c890d113c182bcaa150fe928708e419e254d56108c1df4bee5 not found: ID does not exist"
Jul 16 11:05:33.524853 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:33.524833    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"]
Jul 16 11:05:33.528679 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:33.528656    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-89289a-6cc4f5dc7b-b88p2"]
Jul 16 11:05:33.801846 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:33.801762    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c722a34-6c8b-44f5-bbfa-7e3cc07d7514" path="/var/lib/kubelet/pods/7c722a34-6c8b-44f5-bbfa-7e3cc07d7514/volumes"
Jul 16 11:05:43.026008 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.025975    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"]
Jul 16 11:05:43.026539 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.026515    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c722a34-6c8b-44f5-bbfa-7e3cc07d7514" containerName="maas-api"
Jul 16 11:05:43.026539 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.026533    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c722a34-6c8b-44f5-bbfa-7e3cc07d7514" containerName="maas-api"
Jul 16 11:05:43.026660 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.026565    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56822c92-e464-46d9-873d-d5523722d800" containerName="maas-api"
Jul 16 11:05:43.026660 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.026574    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="56822c92-e464-46d9-873d-d5523722d800" containerName="maas-api"
Jul 16 11:05:43.026825 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.026679    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c722a34-6c8b-44f5-bbfa-7e3cc07d7514" containerName="maas-api"
Jul 16 11:05:43.026825 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.026696    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="56822c92-e464-46d9-873d-d5523722d800" containerName="maas-api"
Jul 16 11:05:43.031092 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.031071    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:05:43.034876 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.034854    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-046938-serving-cert\""
Jul 16 11:05:43.035004 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.034859    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kn7cc\""
Jul 16 11:05:43.035153 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.035140    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 11:05:43.035473 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.035454    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 11:05:43.052088 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.052069    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"]
Jul 16 11:05:43.064859 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.064833    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq26w\" (UniqueName: \"kubernetes.io/projected/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-kube-api-access-sq26w\") pod \"maas-api-e2e-shared-a-046938-7884fc679f-55jsp\" (UID: \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:05:43.064959 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.064904    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-maas-api-tls\") pod \"maas-api-e2e-shared-a-046938-7884fc679f-55jsp\" (UID: \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:05:43.065017 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.065000    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-openshift-service-ca\") pod \"maas-api-e2e-shared-a-046938-7884fc679f-55jsp\" (UID: \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:05:43.165336 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.165310    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq26w\" (UniqueName: \"kubernetes.io/projected/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-kube-api-access-sq26w\") pod \"maas-api-e2e-shared-a-046938-7884fc679f-55jsp\" (UID: \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:05:43.165459 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.165354    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-maas-api-tls\") pod \"maas-api-e2e-shared-a-046938-7884fc679f-55jsp\" (UID: \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:05:43.165459 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.165395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-openshift-service-ca\") pod \"maas-api-e2e-shared-a-046938-7884fc679f-55jsp\" (UID: \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:05:43.166042 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.166025    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-openshift-service-ca\") pod \"maas-api-e2e-shared-a-046938-7884fc679f-55jsp\" (UID: \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:05:43.167894 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.167872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-maas-api-tls\") pod \"maas-api-e2e-shared-a-046938-7884fc679f-55jsp\" (UID: \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:05:43.184373 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.184355    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq26w\" (UniqueName: \"kubernetes.io/projected/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-kube-api-access-sq26w\") pod \"maas-api-e2e-shared-a-046938-7884fc679f-55jsp\" (UID: \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:05:43.341491 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.341410    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:05:43.473802 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.473772    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"]
Jul 16 11:05:43.475688 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:05:43.475659    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ee007e_ed40_4149_b728_8c9aacf0a9ef.slice/crio-b6db65b09d66dbfaba521de760fb74cb3e770ef876d15cf7226c4f64ae10c567 WatchSource:0}: Error finding container b6db65b09d66dbfaba521de760fb74cb3e770ef876d15cf7226c4f64ae10c567: Status 404 returned error can't find the container with id b6db65b09d66dbfaba521de760fb74cb3e770ef876d15cf7226c4f64ae10c567
Jul 16 11:05:43.549053 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:43.549020    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp" event={"ID":"d7ee007e-ed40-4149-b728-8c9aacf0a9ef","Type":"ContainerStarted","Data":"b6db65b09d66dbfaba521de760fb74cb3e770ef876d15cf7226c4f64ae10c567"}
Jul 16 11:05:44.555607 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:44.555574    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp" event={"ID":"d7ee007e-ed40-4149-b728-8c9aacf0a9ef","Type":"ContainerStarted","Data":"9f96edf454ba20f3cbc4d121c49827272afdc0d449aa81dea027ea9ea85a0b5d"}
Jul 16 11:05:44.555982 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:44.555714    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:05:44.585583 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:44.585527    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp" podStartSLOduration=2.15553288 podStartE2EDuration="2.58550892s" totalImagesPullingTime="429.97604ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:05:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:05:44.583339995 +0000 UTC m=+1999.320375265" watchObservedRunningTime="2026-07-16 11:05:44.58550892 +0000 UTC m=+1999.322544192"
Jul 16 11:05:50.569240 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:50.569211    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:05:51.560558 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:51.560524    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"]
Jul 16 11:05:51.563907 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:51.563891    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:05:51.566622 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:51.566600    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-307109-serving-cert\""
Jul 16 11:05:51.596816 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:51.596795    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"]
Jul 16 11:05:51.638712 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:51.638688    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/898d123c-1211-4746-95ed-606cb15efeb8-maas-api-tls\") pod \"maas-api-e2e-shared-b-307109-54dd44994-d6fp8\" (UID: \"898d123c-1211-4746-95ed-606cb15efeb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:05:51.638837 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:51.638729    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/898d123c-1211-4746-95ed-606cb15efeb8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-307109-54dd44994-d6fp8\" (UID: \"898d123c-1211-4746-95ed-606cb15efeb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:05:51.638884 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:51.638857    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6khc\" (UniqueName: \"kubernetes.io/projected/898d123c-1211-4746-95ed-606cb15efeb8-kube-api-access-d6khc\") pod \"maas-api-e2e-shared-b-307109-54dd44994-d6fp8\" (UID: \"898d123c-1211-4746-95ed-606cb15efeb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:05:51.739430 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:51.739404    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/898d123c-1211-4746-95ed-606cb15efeb8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-307109-54dd44994-d6fp8\" (UID: \"898d123c-1211-4746-95ed-606cb15efeb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:05:51.739539 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:51.739463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6khc\" (UniqueName: \"kubernetes.io/projected/898d123c-1211-4746-95ed-606cb15efeb8-kube-api-access-d6khc\") pod \"maas-api-e2e-shared-b-307109-54dd44994-d6fp8\" (UID: \"898d123c-1211-4746-95ed-606cb15efeb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:05:51.739539 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:51.739506    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/898d123c-1211-4746-95ed-606cb15efeb8-maas-api-tls\") pod \"maas-api-e2e-shared-b-307109-54dd44994-d6fp8\" (UID: \"898d123c-1211-4746-95ed-606cb15efeb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:05:51.739625 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:05:51.739611    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-serving-cert: secret "maas-api-e2e-shared-b-307109-serving-cert" not found
Jul 16 11:05:51.739678 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:05:51.739669    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/898d123c-1211-4746-95ed-606cb15efeb8-maas-api-tls podName:898d123c-1211-4746-95ed-606cb15efeb8 nodeName:}" failed. No retries permitted until 2026-07-16 11:05:52.239653852 +0000 UTC m=+2006.976689101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/898d123c-1211-4746-95ed-606cb15efeb8-maas-api-tls") pod "maas-api-e2e-shared-b-307109-54dd44994-d6fp8" (UID: "898d123c-1211-4746-95ed-606cb15efeb8") : secret "maas-api-e2e-shared-b-307109-serving-cert" not found
Jul 16 11:05:51.740139 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:51.740119    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/898d123c-1211-4746-95ed-606cb15efeb8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-307109-54dd44994-d6fp8\" (UID: \"898d123c-1211-4746-95ed-606cb15efeb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:05:51.772016 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:51.771992    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6khc\" (UniqueName: \"kubernetes.io/projected/898d123c-1211-4746-95ed-606cb15efeb8-kube-api-access-d6khc\") pod \"maas-api-e2e-shared-b-307109-54dd44994-d6fp8\" (UID: \"898d123c-1211-4746-95ed-606cb15efeb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:05:52.243797 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:52.243761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/898d123c-1211-4746-95ed-606cb15efeb8-maas-api-tls\") pod \"maas-api-e2e-shared-b-307109-54dd44994-d6fp8\" (UID: \"898d123c-1211-4746-95ed-606cb15efeb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:05:52.246104 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:52.246083    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/898d123c-1211-4746-95ed-606cb15efeb8-maas-api-tls\") pod \"maas-api-e2e-shared-b-307109-54dd44994-d6fp8\" (UID: \"898d123c-1211-4746-95ed-606cb15efeb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:05:52.474542 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:52.474508    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:05:52.607717 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:52.607695    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"]
Jul 16 11:05:52.610059 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:05:52.610036    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod898d123c_1211_4746_95ed_606cb15efeb8.slice/crio-75f8601ef716c3e1bb1e9af7b1cc3734af91ac889cfc7b749e647510277f1870 WatchSource:0}: Error finding container 75f8601ef716c3e1bb1e9af7b1cc3734af91ac889cfc7b749e647510277f1870: Status 404 returned error can't find the container with id 75f8601ef716c3e1bb1e9af7b1cc3734af91ac889cfc7b749e647510277f1870
Jul 16 11:05:53.600698 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:53.600666    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8" event={"ID":"898d123c-1211-4746-95ed-606cb15efeb8","Type":"ContainerStarted","Data":"bd45bc14e232ea37cad57f5fe6708dadd9518aa82f3174a76d894e20bd83be81"}
Jul 16 11:05:53.600874 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:53.600714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8" event={"ID":"898d123c-1211-4746-95ed-606cb15efeb8","Type":"ContainerStarted","Data":"75f8601ef716c3e1bb1e9af7b1cc3734af91ac889cfc7b749e647510277f1870"}
Jul 16 11:05:53.600874 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:53.600776    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:05:53.629781 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:53.629717    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8" podStartSLOduration=2.259372708 podStartE2EDuration="2.629702515s" totalImagesPullingTime="370.329807ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:05:53.627607189 +0000 UTC m=+2008.364642461" watchObservedRunningTime="2026-07-16 11:05:53.629702515 +0000 UTC m=+2008.366737786"
Jul 16 11:05:59.611917 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:05:59.611887    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:06:08.382029 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.381998    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"]
Jul 16 11:06:08.385496 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.385473    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.389611 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.389588    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-046938\"/\"kube-root-ca.crt\""
Jul 16 11:06:08.393150 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.393134    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-046938\"/\"default-dockercfg-9rswc\""
Jul 16 11:06:08.393955 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.393941    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-046938\"/\"auth-test-model-046938-kserve-self-signed-certs\""
Jul 16 11:06:08.394407 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.394396    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-046938\"/\"openshift-service-ca.crt\""
Jul 16 11:06:08.449599 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.449569    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"]
Jul 16 11:06:08.485512 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.485473    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-dshm\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.485711 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.485566    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-home\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.485711 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.485601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-tmp-dir\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.485711 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.485628    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-model-cache\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.485711 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.485649    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecedfca9-e911-4039-b757-24efdebb22dc-tls-certs\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.485711 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.485685    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nbpt\" (UniqueName: \"kubernetes.io/projected/ecedfca9-e911-4039-b757-24efdebb22dc-kube-api-access-2nbpt\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.586859 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.586821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-dshm\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.587065 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.586884    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-home\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.587065 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.586913    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-tmp-dir\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.587065 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.586930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-model-cache\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.587065 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.586947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecedfca9-e911-4039-b757-24efdebb22dc-tls-certs\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.587269 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.587090    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nbpt\" (UniqueName: \"kubernetes.io/projected/ecedfca9-e911-4039-b757-24efdebb22dc-kube-api-access-2nbpt\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.587359 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.587336    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-tmp-dir\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.587423 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.587364    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-model-cache\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.587423 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.587412    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-home\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.589356 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.589337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-dshm\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.589479 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.589462    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecedfca9-e911-4039-b757-24efdebb22dc-tls-certs\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.601222 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.601191    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nbpt\" (UniqueName: \"kubernetes.io/projected/ecedfca9-e911-4039-b757-24efdebb22dc-kube-api-access-2nbpt\") pod \"auth-test-model-046938-kserve-7cf44dfc47-wkjxd\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") " pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.694244 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.694151    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:08.854493 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:08.854454    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"]
Jul 16 11:06:08.858593 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:06:08.858562    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podecedfca9_e911_4039_b757_24efdebb22dc.slice/crio-bb1d9fb9fe8428fbe25474e0fa2efa1e6eec84d29fec2bf2e10cb331cd6e5b68 WatchSource:0}: Error finding container bb1d9fb9fe8428fbe25474e0fa2efa1e6eec84d29fec2bf2e10cb331cd6e5b68: Status 404 returned error can't find the container with id bb1d9fb9fe8428fbe25474e0fa2efa1e6eec84d29fec2bf2e10cb331cd6e5b68
Jul 16 11:06:09.679387 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:09.679353    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd" event={"ID":"ecedfca9-e911-4039-b757-24efdebb22dc","Type":"ContainerStarted","Data":"485314eecdd65468f4a4cbebeaf799ab120d7efba2ea2b836ab0a79171668e08"}
Jul 16 11:06:09.679820 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:09.679398    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd" event={"ID":"ecedfca9-e911-4039-b757-24efdebb22dc","Type":"ContainerStarted","Data":"bb1d9fb9fe8428fbe25474e0fa2efa1e6eec84d29fec2bf2e10cb331cd6e5b68"}
Jul 16 11:06:09.713545 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:09.713493    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd" podStartSLOduration=1.713476788 podStartE2EDuration="1.713476788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:06:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:06:09.712319329 +0000 UTC m=+2024.449354600" watchObservedRunningTime="2026-07-16 11:06:09.713476788 +0000 UTC m=+2024.450512060"
Jul 16 11:06:18.695274 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:18.695239    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:18.695786 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:18.695294    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:18.707318 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:18.707291    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:18.732717 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:18.732690    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:06:38.305002 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.304967    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"]
Jul 16 11:06:38.308394 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.308377    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.311642 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.311608    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-307109\"/\"kube-root-ca.crt\""
Jul 16 11:06:38.311642 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.311634    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-307109\"/\"openshift-service-ca.crt\""
Jul 16 11:06:38.311843 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.311621    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-307109\"/\"auth-test-model-307109-kserve-self-signed-certs\""
Jul 16 11:06:38.311843 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.311769    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-307109\"/\"default-dockercfg-m74vc\""
Jul 16 11:06:38.317378 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.317359    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"]
Jul 16 11:06:38.361589 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.361557    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8n4v\" (UniqueName: \"kubernetes.io/projected/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-kube-api-access-g8n4v\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.361775 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.361604    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-home\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.361775 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.361620    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-dshm\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.361775 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.361757    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-model-cache\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.361915 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.361804    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-tls-certs\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.361915 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.361847    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-tmp-dir\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.463195 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.463159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8n4v\" (UniqueName: \"kubernetes.io/projected/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-kube-api-access-g8n4v\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.463423 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.463203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-home\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.463423 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.463220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-dshm\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.463423 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.463263    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-model-cache\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.463423 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.463287    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-tls-certs\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.463423 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.463325    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-tmp-dir\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.463648 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.463627    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-home\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.463694 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.463648    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-model-cache\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.463732 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.463724    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-tmp-dir\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.465713 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.465682    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-dshm\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.465839 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.465768    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-tls-certs\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.471794 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.471739    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8n4v\" (UniqueName: \"kubernetes.io/projected/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-kube-api-access-g8n4v\") pod \"auth-test-model-307109-kserve-5998d5dc5f-6hk65\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") " pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.618558 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.618519    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:38.753152 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.753126    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"]
Jul 16 11:06:38.754696 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:06:38.754668    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8a0d2b3_b7b1_4161_855b_a06c6fd77370.slice/crio-b2b03994f177e75a1b7c8767e446969c8478deb652d48445f3b31398181592b9 WatchSource:0}: Error finding container b2b03994f177e75a1b7c8767e446969c8478deb652d48445f3b31398181592b9: Status 404 returned error can't find the container with id b2b03994f177e75a1b7c8767e446969c8478deb652d48445f3b31398181592b9
Jul 16 11:06:38.817155 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.817123    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65" event={"ID":"e8a0d2b3-b7b1-4161-855b-a06c6fd77370","Type":"ContainerStarted","Data":"0640c784ebe78e6e20556bbdfbead3ceaaa83115c965acf50bdbf7938457ac94"}
Jul 16 11:06:38.817344 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.817169    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65" event={"ID":"e8a0d2b3-b7b1-4161-855b-a06c6fd77370","Type":"ContainerStarted","Data":"b2b03994f177e75a1b7c8767e446969c8478deb652d48445f3b31398181592b9"}
Jul 16 11:06:38.837217 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:38.837160    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65" podStartSLOduration=0.837142044 podStartE2EDuration="837.142044ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:06:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:06:38.835617351 +0000 UTC m=+2053.572652679" watchObservedRunningTime="2026-07-16 11:06:38.837142044 +0000 UTC m=+2053.574177314"
Jul 16 11:06:48.619634 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:48.619556    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:48.619634 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:48.619600    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:48.632131 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:48.632105    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:06:48.874907 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:06:48.874820    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:07:27.951639 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:07:27.951518    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 11:07:27.961427 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:07:27.961405    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 11:08:25.880816 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:08:25.878471    2576 scope.go:117] "RemoveContainer" containerID="848f7ba9a4ad319aa2af888db14c037d46a2cdbeea880d728517d6e5563038e6"
Jul 16 11:08:47.454595 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:08:47.454557    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"]
Jul 16 11:08:48.352731 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:08:48.352695    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"]
Jul 16 11:09:02.475315 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.475270    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd" podUID="ecedfca9-e911-4039-b757-24efdebb22dc" containerName="main" containerID="cri-o://485314eecdd65468f4a4cbebeaf799ab120d7efba2ea2b836ab0a79171668e08" gracePeriod=45
Jul 16 11:09:02.730198 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.730136    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:09:02.780580 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.780544    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nbpt\" (UniqueName: \"kubernetes.io/projected/ecedfca9-e911-4039-b757-24efdebb22dc-kube-api-access-2nbpt\") pod \"ecedfca9-e911-4039-b757-24efdebb22dc\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") "
Jul 16 11:09:02.780771 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.780628    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-tmp-dir\") pod \"ecedfca9-e911-4039-b757-24efdebb22dc\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") "
Jul 16 11:09:02.780771 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.780669    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-dshm\") pod \"ecedfca9-e911-4039-b757-24efdebb22dc\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") "
Jul 16 11:09:02.780771 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.780720    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-home\") pod \"ecedfca9-e911-4039-b757-24efdebb22dc\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") "
Jul 16 11:09:02.780926 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.780784    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-model-cache\") pod \"ecedfca9-e911-4039-b757-24efdebb22dc\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") "
Jul 16 11:09:02.780926 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.780840    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecedfca9-e911-4039-b757-24efdebb22dc-tls-certs\") pod \"ecedfca9-e911-4039-b757-24efdebb22dc\" (UID: \"ecedfca9-e911-4039-b757-24efdebb22dc\") "
Jul 16 11:09:02.781034 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.780938    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ecedfca9-e911-4039-b757-24efdebb22dc" (UID: "ecedfca9-e911-4039-b757-24efdebb22dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:09:02.781087 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.781033    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-home" (OuterVolumeSpecName: "home") pod "ecedfca9-e911-4039-b757-24efdebb22dc" (UID: "ecedfca9-e911-4039-b757-24efdebb22dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:09:02.781143 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.781071    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "ecedfca9-e911-4039-b757-24efdebb22dc" (UID: "ecedfca9-e911-4039-b757-24efdebb22dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:09:02.781233 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.781209    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-tmp-dir\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:09:02.781233 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.781236    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-home\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:09:02.781425 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.781247    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-model-cache\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:09:02.782925 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.782903    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-dshm" (OuterVolumeSpecName: "dshm") pod "ecedfca9-e911-4039-b757-24efdebb22dc" (UID: "ecedfca9-e911-4039-b757-24efdebb22dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:09:02.783156 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.783136    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecedfca9-e911-4039-b757-24efdebb22dc-kube-api-access-2nbpt" (OuterVolumeSpecName: "kube-api-access-2nbpt") pod "ecedfca9-e911-4039-b757-24efdebb22dc" (UID: "ecedfca9-e911-4039-b757-24efdebb22dc"). InnerVolumeSpecName "kube-api-access-2nbpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:09:02.783241 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.783186    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecedfca9-e911-4039-b757-24efdebb22dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ecedfca9-e911-4039-b757-24efdebb22dc" (UID: "ecedfca9-e911-4039-b757-24efdebb22dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:09:02.882585 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.882551    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecedfca9-e911-4039-b757-24efdebb22dc-dshm\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:09:02.882585 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.882579    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecedfca9-e911-4039-b757-24efdebb22dc-tls-certs\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:09:02.882585 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:02.882589    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nbpt\" (UniqueName: \"kubernetes.io/projected/ecedfca9-e911-4039-b757-24efdebb22dc-kube-api-access-2nbpt\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:09:03.372909 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.372867    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65" podUID="e8a0d2b3-b7b1-4161-855b-a06c6fd77370" containerName="main" containerID="cri-o://0640c784ebe78e6e20556bbdfbead3ceaaa83115c965acf50bdbf7938457ac94" gracePeriod=45
Jul 16 11:09:03.495929 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.495898    2576 generic.go:356] "Generic (PLEG): container finished" podID="ecedfca9-e911-4039-b757-24efdebb22dc" containerID="485314eecdd65468f4a4cbebeaf799ab120d7efba2ea2b836ab0a79171668e08" exitCode=0
Jul 16 11:09:03.496306 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.495941    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"
Jul 16 11:09:03.496306 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.495972    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd" event={"ID":"ecedfca9-e911-4039-b757-24efdebb22dc","Type":"ContainerDied","Data":"485314eecdd65468f4a4cbebeaf799ab120d7efba2ea2b836ab0a79171668e08"}
Jul 16 11:09:03.496306 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.496012    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd" event={"ID":"ecedfca9-e911-4039-b757-24efdebb22dc","Type":"ContainerDied","Data":"bb1d9fb9fe8428fbe25474e0fa2efa1e6eec84d29fec2bf2e10cb331cd6e5b68"}
Jul 16 11:09:03.496306 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.496029    2576 scope.go:117] "RemoveContainer" containerID="485314eecdd65468f4a4cbebeaf799ab120d7efba2ea2b836ab0a79171668e08"
Jul 16 11:09:03.497971 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.497952    2576 generic.go:356] "Generic (PLEG): container finished" podID="e8a0d2b3-b7b1-4161-855b-a06c6fd77370" containerID="0640c784ebe78e6e20556bbdfbead3ceaaa83115c965acf50bdbf7938457ac94" exitCode=0
Jul 16 11:09:03.498087 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.497978    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65" event={"ID":"e8a0d2b3-b7b1-4161-855b-a06c6fd77370","Type":"ContainerDied","Data":"0640c784ebe78e6e20556bbdfbead3ceaaa83115c965acf50bdbf7938457ac94"}
Jul 16 11:09:03.505942 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.505925    2576 scope.go:117] "RemoveContainer" containerID="485314eecdd65468f4a4cbebeaf799ab120d7efba2ea2b836ab0a79171668e08"
Jul 16 11:09:03.506224 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:09:03.506206    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"485314eecdd65468f4a4cbebeaf799ab120d7efba2ea2b836ab0a79171668e08\": container with ID starting with 485314eecdd65468f4a4cbebeaf799ab120d7efba2ea2b836ab0a79171668e08 not found: ID does not exist" containerID="485314eecdd65468f4a4cbebeaf799ab120d7efba2ea2b836ab0a79171668e08"
Jul 16 11:09:03.506293 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.506235    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"485314eecdd65468f4a4cbebeaf799ab120d7efba2ea2b836ab0a79171668e08"} err="failed to get container status \"485314eecdd65468f4a4cbebeaf799ab120d7efba2ea2b836ab0a79171668e08\": rpc error: code = NotFound desc = could not find container \"485314eecdd65468f4a4cbebeaf799ab120d7efba2ea2b836ab0a79171668e08\": container with ID starting with 485314eecdd65468f4a4cbebeaf799ab120d7efba2ea2b836ab0a79171668e08 not found: ID does not exist"
Jul 16 11:09:03.519486 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.519455    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"]
Jul 16 11:09:03.524444 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.524415    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-046938/auth-test-model-046938-kserve-7cf44dfc47-wkjxd"]
Jul 16 11:09:03.632000 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.631940    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:09:03.690884 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.690850    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-home\") pod \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") "
Jul 16 11:09:03.690884 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.690885    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-dshm\") pod \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") "
Jul 16 11:09:03.691140 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.690918    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-tmp-dir\") pod \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") "
Jul 16 11:09:03.691140 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.690957    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-model-cache\") pod \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") "
Jul 16 11:09:03.691140 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.690995    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8n4v\" (UniqueName: \"kubernetes.io/projected/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-kube-api-access-g8n4v\") pod \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") "
Jul 16 11:09:03.691140 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.691016    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-tls-certs\") pod \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\" (UID: \"e8a0d2b3-b7b1-4161-855b-a06c6fd77370\") "
Jul 16 11:09:03.691332 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.691182    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-home" (OuterVolumeSpecName: "home") pod "e8a0d2b3-b7b1-4161-855b-a06c6fd77370" (UID: "e8a0d2b3-b7b1-4161-855b-a06c6fd77370"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:09:03.691332 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.691225    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-model-cache" (OuterVolumeSpecName: "model-cache") pod "e8a0d2b3-b7b1-4161-855b-a06c6fd77370" (UID: "e8a0d2b3-b7b1-4161-855b-a06c6fd77370"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:09:03.691332 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.691239    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e8a0d2b3-b7b1-4161-855b-a06c6fd77370" (UID: "e8a0d2b3-b7b1-4161-855b-a06c6fd77370"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:09:03.693202 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.693170    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e8a0d2b3-b7b1-4161-855b-a06c6fd77370" (UID: "e8a0d2b3-b7b1-4161-855b-a06c6fd77370"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:09:03.693306 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.693229    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-dshm" (OuterVolumeSpecName: "dshm") pod "e8a0d2b3-b7b1-4161-855b-a06c6fd77370" (UID: "e8a0d2b3-b7b1-4161-855b-a06c6fd77370"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:09:03.693306 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.693250    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-kube-api-access-g8n4v" (OuterVolumeSpecName: "kube-api-access-g8n4v") pod "e8a0d2b3-b7b1-4161-855b-a06c6fd77370" (UID: "e8a0d2b3-b7b1-4161-855b-a06c6fd77370"). InnerVolumeSpecName "kube-api-access-g8n4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:09:03.791964 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.791933    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-tls-certs\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:09:03.791964 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.791959    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-home\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:09:03.791964 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.791967    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-dshm\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:09:03.792239 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.791978    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-tmp-dir\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:09:03.792239 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.791991    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-model-cache\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:09:03.792239 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.792003    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8n4v\" (UniqueName: \"kubernetes.io/projected/e8a0d2b3-b7b1-4161-855b-a06c6fd77370-kube-api-access-g8n4v\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:09:03.801466 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:03.801439    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecedfca9-e911-4039-b757-24efdebb22dc" path="/var/lib/kubelet/pods/ecedfca9-e911-4039-b757-24efdebb22dc/volumes"
Jul 16 11:09:04.509338 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:04.509302    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65" event={"ID":"e8a0d2b3-b7b1-4161-855b-a06c6fd77370","Type":"ContainerDied","Data":"b2b03994f177e75a1b7c8767e446969c8478deb652d48445f3b31398181592b9"}
Jul 16 11:09:04.509836 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:04.509351    2576 scope.go:117] "RemoveContainer" containerID="0640c784ebe78e6e20556bbdfbead3ceaaa83115c965acf50bdbf7938457ac94"
Jul 16 11:09:04.509836 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:04.509317    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"
Jul 16 11:09:04.529079 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:04.529045    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"]
Jul 16 11:09:04.537590 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:04.537558    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-307109/auth-test-model-307109-kserve-5998d5dc5f-6hk65"]
Jul 16 11:09:05.801367 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:05.801334    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8a0d2b3-b7b1-4161-855b-a06c6fd77370" path="/var/lib/kubelet/pods/e8a0d2b3-b7b1-4161-855b-a06c6fd77370/volumes"
Jul 16 11:09:24.041270 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.041237    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"]
Jul 16 11:09:24.041675 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.041586    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecedfca9-e911-4039-b757-24efdebb22dc" containerName="main"
Jul 16 11:09:24.041675 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.041597    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecedfca9-e911-4039-b757-24efdebb22dc" containerName="main"
Jul 16 11:09:24.041675 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.041610    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8a0d2b3-b7b1-4161-855b-a06c6fd77370" containerName="main"
Jul 16 11:09:24.041675 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.041615    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8a0d2b3-b7b1-4161-855b-a06c6fd77370" containerName="main"
Jul 16 11:09:24.041857 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.041678    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8a0d2b3-b7b1-4161-855b-a06c6fd77370" containerName="main"
Jul 16 11:09:24.041857 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.041691    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecedfca9-e911-4039-b757-24efdebb22dc" containerName="main"
Jul 16 11:09:24.046101 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.046083    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.052733 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.052710    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-307109\"/\"openshift-service-ca.crt\""
Jul 16 11:09:24.052733 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.052722    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-307109\"/\"default-dockercfg-m74vc\""
Jul 16 11:09:24.053157 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.053143    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-307109\"/\"sub-test-model-307109-kserve-self-signed-certs\""
Jul 16 11:09:24.055427 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.055414    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-307109\"/\"kube-root-ca.crt\""
Jul 16 11:09:24.072041 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.072013    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"]
Jul 16 11:09:24.180141 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.180106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/246adb6b-832b-4164-a876-070597fb1bb2-tls-certs\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.180141 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.180158    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-home\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.180462 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.180184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-dshm\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.180462 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.180270    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8npdn\" (UniqueName: \"kubernetes.io/projected/246adb6b-832b-4164-a876-070597fb1bb2-kube-api-access-8npdn\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.180462 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.180312    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-tmp-dir\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.180462 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.180359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-model-cache\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.281448 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.281411    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8npdn\" (UniqueName: \"kubernetes.io/projected/246adb6b-832b-4164-a876-070597fb1bb2-kube-api-access-8npdn\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.281652 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.281457    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-tmp-dir\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.281652 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.281482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-model-cache\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.281652 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.281529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/246adb6b-832b-4164-a876-070597fb1bb2-tls-certs\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.281652 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.281553    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-home\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.281652 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.281568    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-dshm\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.281976 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.281927    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-tmp-dir\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.281976 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.281965    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-model-cache\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.282093 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.281978    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-home\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.283910 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.283854    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-dshm\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.284049 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.284033    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/246adb6b-832b-4164-a876-070597fb1bb2-tls-certs\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.293491 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.293439    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8npdn\" (UniqueName: \"kubernetes.io/projected/246adb6b-832b-4164-a876-070597fb1bb2-kube-api-access-8npdn\") pod \"sub-test-model-307109-kserve-57479ddb9f-tvrww\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") " pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.355524 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.355488    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:24.495450 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.495425    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"]
Jul 16 11:09:24.497350 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:09:24.497325    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod246adb6b_832b_4164_a876_070597fb1bb2.slice/crio-993f76151aee8b3a6d5ec17e2c7eeb9f8cd0e4eb89ba17c85fadb2f41d00ac1e WatchSource:0}: Error finding container 993f76151aee8b3a6d5ec17e2c7eeb9f8cd0e4eb89ba17c85fadb2f41d00ac1e: Status 404 returned error can't find the container with id 993f76151aee8b3a6d5ec17e2c7eeb9f8cd0e4eb89ba17c85fadb2f41d00ac1e
Jul 16 11:09:24.606914 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.606877    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww" event={"ID":"246adb6b-832b-4164-a876-070597fb1bb2","Type":"ContainerStarted","Data":"2982d815deaf1bb6a280b849c6ce9cbb3ae5c238158d48f1607401481ac349c8"}
Jul 16 11:09:24.606914 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.606927    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww" event={"ID":"246adb6b-832b-4164-a876-070597fb1bb2","Type":"ContainerStarted","Data":"993f76151aee8b3a6d5ec17e2c7eeb9f8cd0e4eb89ba17c85fadb2f41d00ac1e"}
Jul 16 11:09:24.631506 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:24.631447    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww" podStartSLOduration=0.63142661 podStartE2EDuration="631.42661ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:09:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:09:24.631263225 +0000 UTC m=+2219.368298497" watchObservedRunningTime="2026-07-16 11:09:24.63142661 +0000 UTC m=+2219.368461882"
Jul 16 11:09:34.356421 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:34.356372    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:34.356421 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:34.356423    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:34.369298 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:34.369274    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:09:34.669782 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:09:34.669684    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:10:40.319542 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:40.319510    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"]
Jul 16 11:10:54.674935 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.674898    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"]
Jul 16 11:10:54.678273 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.678254    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.704711 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.704688    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-046938\"/\"default-dockercfg-9rswc\""
Jul 16 11:10:54.706517 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.706502    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-046938\"/\"openshift-service-ca.crt\""
Jul 16 11:10:54.708697 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.708680    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-046938\"/\"kube-root-ca.crt\""
Jul 16 11:10:54.712063 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.712044    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-046938\"/\"rate-test-model-046938-kserve-self-signed-certs\""
Jul 16 11:10:54.713387 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.713368    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"]
Jul 16 11:10:54.719910 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.719891    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-tls-certs\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.720021 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.719928    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwwps\" (UniqueName: \"kubernetes.io/projected/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-kube-api-access-vwwps\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.720085 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.720047    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-model-cache\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.720142 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.720094    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-tmp-dir\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.720193 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.720164    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-home\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.720231 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.720211    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-dshm\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.821209 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.821173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-home\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.821209 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.821209    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-dshm\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.821445 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.821302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-tls-certs\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.821445 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.821341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwwps\" (UniqueName: \"kubernetes.io/projected/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-kube-api-access-vwwps\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.821445 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.821415    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-model-cache\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.821616 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.821601    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-home\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.821694 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.821666    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-tmp-dir\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.821861 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.821840    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-model-cache\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.822000 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.821981    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-tmp-dir\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.823991 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.823971    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-dshm\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.824209 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.824189    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-tls-certs\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.833950 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.833925    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwwps\" (UniqueName: \"kubernetes.io/projected/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-kube-api-access-vwwps\") pod \"rate-test-model-046938-kserve-6f67ff9bf9-48gch\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") " pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:54.987685 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:54.987592    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:10:55.145425 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:10:55.145395    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1b45c9c_cbf9_4f0a_8a99_3220f5100f6b.slice/crio-3b52916874f92f0efa68a01f61b24522eb7e01d5f58202d57075d3a78fbb2602 WatchSource:0}: Error finding container 3b52916874f92f0efa68a01f61b24522eb7e01d5f58202d57075d3a78fbb2602: Status 404 returned error can't find the container with id 3b52916874f92f0efa68a01f61b24522eb7e01d5f58202d57075d3a78fbb2602
Jul 16 11:10:55.149159 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.149141    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"]
Jul 16 11:10:55.339013 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.338907    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww" podUID="246adb6b-832b-4164-a876-070597fb1bb2" containerName="main" containerID="cri-o://2982d815deaf1bb6a280b849c6ce9cbb3ae5c238158d48f1607401481ac349c8" gracePeriod=45
Jul 16 11:10:55.571474 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.571446    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:10:55.628454 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.628422    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-model-cache\") pod \"246adb6b-832b-4164-a876-070597fb1bb2\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") "
Jul 16 11:10:55.628642 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.628469    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/246adb6b-832b-4164-a876-070597fb1bb2-tls-certs\") pod \"246adb6b-832b-4164-a876-070597fb1bb2\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") "
Jul 16 11:10:55.628642 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.628497    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-tmp-dir\") pod \"246adb6b-832b-4164-a876-070597fb1bb2\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") "
Jul 16 11:10:55.628642 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.628523    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-home\") pod \"246adb6b-832b-4164-a876-070597fb1bb2\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") "
Jul 16 11:10:55.628642 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.628557    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-dshm\") pod \"246adb6b-832b-4164-a876-070597fb1bb2\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") "
Jul 16 11:10:55.628901 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.628643    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8npdn\" (UniqueName: \"kubernetes.io/projected/246adb6b-832b-4164-a876-070597fb1bb2-kube-api-access-8npdn\") pod \"246adb6b-832b-4164-a876-070597fb1bb2\" (UID: \"246adb6b-832b-4164-a876-070597fb1bb2\") "
Jul 16 11:10:55.628901 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.628812    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-home" (OuterVolumeSpecName: "home") pod "246adb6b-832b-4164-a876-070597fb1bb2" (UID: "246adb6b-832b-4164-a876-070597fb1bb2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:10:55.629116 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.629047    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "246adb6b-832b-4164-a876-070597fb1bb2" (UID: "246adb6b-832b-4164-a876-070597fb1bb2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:10:55.629116 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.629078    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-model-cache" (OuterVolumeSpecName: "model-cache") pod "246adb6b-832b-4164-a876-070597fb1bb2" (UID: "246adb6b-832b-4164-a876-070597fb1bb2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:10:55.629354 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.629335    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-model-cache\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:10:55.629430 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.629356    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-tmp-dir\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:10:55.629430 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.629368    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-home\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:10:55.630854 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.630825    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-dshm" (OuterVolumeSpecName: "dshm") pod "246adb6b-832b-4164-a876-070597fb1bb2" (UID: "246adb6b-832b-4164-a876-070597fb1bb2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:10:55.631163 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.631144    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/246adb6b-832b-4164-a876-070597fb1bb2-kube-api-access-8npdn" (OuterVolumeSpecName: "kube-api-access-8npdn") pod "246adb6b-832b-4164-a876-070597fb1bb2" (UID: "246adb6b-832b-4164-a876-070597fb1bb2"). InnerVolumeSpecName "kube-api-access-8npdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:10:55.631241 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.631218    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/246adb6b-832b-4164-a876-070597fb1bb2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "246adb6b-832b-4164-a876-070597fb1bb2" (UID: "246adb6b-832b-4164-a876-070597fb1bb2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:10:55.730317 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.730281    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8npdn\" (UniqueName: \"kubernetes.io/projected/246adb6b-832b-4164-a876-070597fb1bb2-kube-api-access-8npdn\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:10:55.730317 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.730311    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/246adb6b-832b-4164-a876-070597fb1bb2-tls-certs\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:10:55.730317 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:55.730322    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/246adb6b-832b-4164-a876-070597fb1bb2-dshm\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:10:56.037915 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:56.037812    2576 generic.go:356] "Generic (PLEG): container finished" podID="246adb6b-832b-4164-a876-070597fb1bb2" containerID="2982d815deaf1bb6a280b849c6ce9cbb3ae5c238158d48f1607401481ac349c8" exitCode=0
Jul 16 11:10:56.037915 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:56.037880    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"
Jul 16 11:10:56.038154 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:56.037905    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww" event={"ID":"246adb6b-832b-4164-a876-070597fb1bb2","Type":"ContainerDied","Data":"2982d815deaf1bb6a280b849c6ce9cbb3ae5c238158d48f1607401481ac349c8"}
Jul 16 11:10:56.038154 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:56.037957    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww" event={"ID":"246adb6b-832b-4164-a876-070597fb1bb2","Type":"ContainerDied","Data":"993f76151aee8b3a6d5ec17e2c7eeb9f8cd0e4eb89ba17c85fadb2f41d00ac1e"}
Jul 16 11:10:56.038154 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:56.037980    2576 scope.go:117] "RemoveContainer" containerID="2982d815deaf1bb6a280b849c6ce9cbb3ae5c238158d48f1607401481ac349c8"
Jul 16 11:10:56.039710 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:56.039618    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch" event={"ID":"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b","Type":"ContainerStarted","Data":"0fa73ed76b5cd181166f41a4fdf22479564b620df4192965771640640456f4d0"}
Jul 16 11:10:56.039710 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:56.039659    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch" event={"ID":"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b","Type":"ContainerStarted","Data":"3b52916874f92f0efa68a01f61b24522eb7e01d5f58202d57075d3a78fbb2602"}
Jul 16 11:10:56.047366 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:56.047348    2576 scope.go:117] "RemoveContainer" containerID="2982d815deaf1bb6a280b849c6ce9cbb3ae5c238158d48f1607401481ac349c8"
Jul 16 11:10:56.047629 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:10:56.047606    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2982d815deaf1bb6a280b849c6ce9cbb3ae5c238158d48f1607401481ac349c8\": container with ID starting with 2982d815deaf1bb6a280b849c6ce9cbb3ae5c238158d48f1607401481ac349c8 not found: ID does not exist" containerID="2982d815deaf1bb6a280b849c6ce9cbb3ae5c238158d48f1607401481ac349c8"
Jul 16 11:10:56.047688 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:56.047639    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2982d815deaf1bb6a280b849c6ce9cbb3ae5c238158d48f1607401481ac349c8"} err="failed to get container status \"2982d815deaf1bb6a280b849c6ce9cbb3ae5c238158d48f1607401481ac349c8\": rpc error: code = NotFound desc = could not find container \"2982d815deaf1bb6a280b849c6ce9cbb3ae5c238158d48f1607401481ac349c8\": container with ID starting with 2982d815deaf1bb6a280b849c6ce9cbb3ae5c238158d48f1607401481ac349c8 not found: ID does not exist"
Jul 16 11:10:56.065506 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:56.065477    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"]
Jul 16 11:10:56.076262 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:56.076239    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-307109/sub-test-model-307109-kserve-57479ddb9f-tvrww"]
Jul 16 11:10:57.801014 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:10:57.800919    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="246adb6b-832b-4164-a876-070597fb1bb2" path="/var/lib/kubelet/pods/246adb6b-832b-4164-a876-070597fb1bb2/volumes"
Jul 16 11:11:04.988575 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:04.988527    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:11:04.989066 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:04.988605    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:11:05.001232 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:05.001208    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:11:05.029049 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:05.029002    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch" podStartSLOduration=11.028985589 podStartE2EDuration="11.028985589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:10:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:10:56.110508195 +0000 UTC m=+2310.847543466" watchObservedRunningTime="2026-07-16 11:11:05.028985589 +0000 UTC m=+2319.766020856"
Jul 16 11:11:05.090986 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:05.090958    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:11:27.530454 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.530421    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"]
Jul 16 11:11:27.530861 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.530819    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="246adb6b-832b-4164-a876-070597fb1bb2" containerName="main"
Jul 16 11:11:27.530861 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.530830    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="246adb6b-832b-4164-a876-070597fb1bb2" containerName="main"
Jul 16 11:11:27.530937 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.530891    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="246adb6b-832b-4164-a876-070597fb1bb2" containerName="main"
Jul 16 11:11:27.535147 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.535132    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.538562 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.538540    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-307109\"/\"rate-test-model-307109-kserve-self-signed-certs\""
Jul 16 11:11:27.538678 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.538557    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-307109\"/\"default-dockercfg-m74vc\""
Jul 16 11:11:27.538678 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.538543    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-307109\"/\"openshift-service-ca.crt\""
Jul 16 11:11:27.538678 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.538591    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-307109\"/\"kube-root-ca.crt\""
Jul 16 11:11:27.546093 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.546072    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"]
Jul 16 11:11:27.625133 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.625098    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-dshm\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.625133 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.625134    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-home\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.625362 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.625160    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx98r\" (UniqueName: \"kubernetes.io/projected/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-kube-api-access-rx98r\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.625362 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.625277    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-model-cache\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.625362 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.625331    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-tls-certs\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.625362 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.625357    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-tmp-dir\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.726313 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.726271    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx98r\" (UniqueName: \"kubernetes.io/projected/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-kube-api-access-rx98r\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.726507 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.726322    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-model-cache\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.726507 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.726360    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-tls-certs\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.726507 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.726388    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-tmp-dir\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.726507 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.726487    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-dshm\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.726727 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.726513    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-home\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.726824 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.726799    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-model-cache\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.726889 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.726833    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-tmp-dir\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.726944 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.726905    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-home\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.728688 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.728672    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-dshm\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.729047 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.729027    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-tls-certs\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.736355 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.736335    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx98r\" (UniqueName: \"kubernetes.io/projected/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-kube-api-access-rx98r\") pod \"rate-test-model-307109-kserve-5c9dbbc49f-ljk25\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") " pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.844494 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.844394    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:27.978153 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:27.978127    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"]
Jul 16 11:11:27.980125 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:11:27.980098    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00ab8772_c7e8_49fc_9eaa_84f6e6f4d82b.slice/crio-8d34fffae422cbe7b59a1f1da8f662aa1f5b8c3a6052ccf811e0802b66724559 WatchSource:0}: Error finding container 8d34fffae422cbe7b59a1f1da8f662aa1f5b8c3a6052ccf811e0802b66724559: Status 404 returned error can't find the container with id 8d34fffae422cbe7b59a1f1da8f662aa1f5b8c3a6052ccf811e0802b66724559
Jul 16 11:11:28.187184 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:28.187140    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25" event={"ID":"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b","Type":"ContainerStarted","Data":"553fcf7f2f80943f4f8e666f32ffd779f73854c1bd611f32f628e0b09a952e9a"}
Jul 16 11:11:28.187184 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:28.187189    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25" event={"ID":"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b","Type":"ContainerStarted","Data":"8d34fffae422cbe7b59a1f1da8f662aa1f5b8c3a6052ccf811e0802b66724559"}
Jul 16 11:11:28.210318 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:28.210269    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25" podStartSLOduration=1.210252637 podStartE2EDuration="1.210252637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:11:28.20923513 +0000 UTC m=+2342.946270401" watchObservedRunningTime="2026-07-16 11:11:28.210252637 +0000 UTC m=+2342.947287910"
Jul 16 11:11:37.845433 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:37.845392    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:37.845858 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:37.845448    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:37.858012 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:37.857985    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:11:38.249963 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:11:38.249937    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:12:17.497822 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:17.497788    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"]
Jul 16 11:12:18.410005 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:18.409968    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"]
Jul 16 11:12:27.980178 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:27.980063    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 11:12:27.989551 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:27.989530    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pfmf_5a80df04-e68b-404c-9ed7-96de4566f8a0/ovn-acl-logging/0.log"
Jul 16 11:12:32.518147 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.518060    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch" podUID="b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b" containerName="main" containerID="cri-o://0fa73ed76b5cd181166f41a4fdf22479564b620df4192965771640640456f4d0" gracePeriod=45
Jul 16 11:12:32.766617 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.766593    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:12:32.841438 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.841360    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-dshm\") pod \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") "
Jul 16 11:12:32.841438 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.841416    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwwps\" (UniqueName: \"kubernetes.io/projected/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-kube-api-access-vwwps\") pod \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") "
Jul 16 11:12:32.841438 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.841433    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-model-cache\") pod \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") "
Jul 16 11:12:32.841729 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.841545    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-home\") pod \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") "
Jul 16 11:12:32.841729 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.841587    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-tls-certs\") pod \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") "
Jul 16 11:12:32.841729 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.841629    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-tmp-dir\") pod \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\" (UID: \"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b\") "
Jul 16 11:12:32.841890 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.841813    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-model-cache" (OuterVolumeSpecName: "model-cache") pod "b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b" (UID: "b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:12:32.841890 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.841831    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-home" (OuterVolumeSpecName: "home") pod "b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b" (UID: "b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:12:32.841974 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.841949    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b" (UID: "b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:12:32.842018 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.841977    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-model-cache\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:32.842018 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.841997    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-home\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:32.843772 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.843732    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b" (UID: "b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:12:32.844031 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.844012    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-dshm" (OuterVolumeSpecName: "dshm") pod "b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b" (UID: "b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:12:32.844105 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.844038    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-kube-api-access-vwwps" (OuterVolumeSpecName: "kube-api-access-vwwps") pod "b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b" (UID: "b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b"). InnerVolumeSpecName "kube-api-access-vwwps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:12:32.942463 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.942433    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-tmp-dir\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:32.942463 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.942462    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-dshm\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:32.942463 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.942471    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwwps\" (UniqueName: \"kubernetes.io/projected/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-kube-api-access-vwwps\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:32.942678 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:32.942480    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b-tls-certs\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:33.429771 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.429681    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25" podUID="00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b" containerName="main" containerID="cri-o://553fcf7f2f80943f4f8e666f32ffd779f73854c1bd611f32f628e0b09a952e9a" gracePeriod=45
Jul 16 11:12:33.510089 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.510056    2576 generic.go:356] "Generic (PLEG): container finished" podID="b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b" containerID="0fa73ed76b5cd181166f41a4fdf22479564b620df4192965771640640456f4d0" exitCode=0
Jul 16 11:12:33.510264 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.510094    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch" event={"ID":"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b","Type":"ContainerDied","Data":"0fa73ed76b5cd181166f41a4fdf22479564b620df4192965771640640456f4d0"}
Jul 16 11:12:33.510264 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.510098    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"
Jul 16 11:12:33.510264 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.510126    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch" event={"ID":"b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b","Type":"ContainerDied","Data":"3b52916874f92f0efa68a01f61b24522eb7e01d5f58202d57075d3a78fbb2602"}
Jul 16 11:12:33.510264 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.510148    2576 scope.go:117] "RemoveContainer" containerID="0fa73ed76b5cd181166f41a4fdf22479564b620df4192965771640640456f4d0"
Jul 16 11:12:33.538328 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.538288    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"]
Jul 16 11:12:33.540939 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.540912    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-046938/rate-test-model-046938-kserve-6f67ff9bf9-48gch"]
Jul 16 11:12:33.542516 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.542406    2576 scope.go:117] "RemoveContainer" containerID="0fa73ed76b5cd181166f41a4fdf22479564b620df4192965771640640456f4d0"
Jul 16 11:12:33.542776 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:12:33.542757    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fa73ed76b5cd181166f41a4fdf22479564b620df4192965771640640456f4d0\": container with ID starting with 0fa73ed76b5cd181166f41a4fdf22479564b620df4192965771640640456f4d0 not found: ID does not exist" containerID="0fa73ed76b5cd181166f41a4fdf22479564b620df4192965771640640456f4d0"
Jul 16 11:12:33.542841 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.542784    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fa73ed76b5cd181166f41a4fdf22479564b620df4192965771640640456f4d0"} err="failed to get container status \"0fa73ed76b5cd181166f41a4fdf22479564b620df4192965771640640456f4d0\": rpc error: code = NotFound desc = could not find container \"0fa73ed76b5cd181166f41a4fdf22479564b620df4192965771640640456f4d0\": container with ID starting with 0fa73ed76b5cd181166f41a4fdf22479564b620df4192965771640640456f4d0 not found: ID does not exist"
Jul 16 11:12:33.687122 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.687060    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:12:33.748850 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.748818    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-tls-certs\") pod \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") "
Jul 16 11:12:33.749055 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.748880    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-dshm\") pod \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") "
Jul 16 11:12:33.749055 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.748908    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-home\") pod \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") "
Jul 16 11:12:33.749055 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.748951    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-tmp-dir\") pod \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") "
Jul 16 11:12:33.749055 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.748996    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx98r\" (UniqueName: \"kubernetes.io/projected/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-kube-api-access-rx98r\") pod \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") "
Jul 16 11:12:33.749055 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.749029    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-model-cache\") pod \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\" (UID: \"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b\") "
Jul 16 11:12:33.749306 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.749263    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-home" (OuterVolumeSpecName: "home") pod "00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b" (UID: "00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:12:33.749306 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.749256    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b" (UID: "00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:12:33.749407 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.749378    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-model-cache" (OuterVolumeSpecName: "model-cache") pod "00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b" (UID: "00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:12:33.749407 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.749398    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-home\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:33.749507 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.749412    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-tmp-dir\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:33.751229 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.751195    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-kube-api-access-rx98r" (OuterVolumeSpecName: "kube-api-access-rx98r") pod "00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b" (UID: "00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b"). InnerVolumeSpecName "kube-api-access-rx98r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:12:33.751486 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.751466    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-dshm" (OuterVolumeSpecName: "dshm") pod "00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b" (UID: "00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:12:33.751486 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.751476    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b" (UID: "00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:12:33.801525 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.801496    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b" path="/var/lib/kubelet/pods/b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b/volumes"
Jul 16 11:12:33.850668 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.850640    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-tls-certs\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:33.850668 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.850664    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-dshm\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:33.850668 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.850673    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx98r\" (UniqueName: \"kubernetes.io/projected/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-kube-api-access-rx98r\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:33.850901 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:33.850682    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b-model-cache\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:34.516044 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:34.516011    2576 generic.go:356] "Generic (PLEG): container finished" podID="00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b" containerID="553fcf7f2f80943f4f8e666f32ffd779f73854c1bd611f32f628e0b09a952e9a" exitCode=0
Jul 16 11:12:34.516229 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:34.516055    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"
Jul 16 11:12:34.516229 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:34.516057    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25" event={"ID":"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b","Type":"ContainerDied","Data":"553fcf7f2f80943f4f8e666f32ffd779f73854c1bd611f32f628e0b09a952e9a"}
Jul 16 11:12:34.516229 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:34.516180    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25" event={"ID":"00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b","Type":"ContainerDied","Data":"8d34fffae422cbe7b59a1f1da8f662aa1f5b8c3a6052ccf811e0802b66724559"}
Jul 16 11:12:34.516229 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:34.516213    2576 scope.go:117] "RemoveContainer" containerID="553fcf7f2f80943f4f8e666f32ffd779f73854c1bd611f32f628e0b09a952e9a"
Jul 16 11:12:34.524246 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:34.524225    2576 scope.go:117] "RemoveContainer" containerID="553fcf7f2f80943f4f8e666f32ffd779f73854c1bd611f32f628e0b09a952e9a"
Jul 16 11:12:34.524484 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:12:34.524469    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"553fcf7f2f80943f4f8e666f32ffd779f73854c1bd611f32f628e0b09a952e9a\": container with ID starting with 553fcf7f2f80943f4f8e666f32ffd779f73854c1bd611f32f628e0b09a952e9a not found: ID does not exist" containerID="553fcf7f2f80943f4f8e666f32ffd779f73854c1bd611f32f628e0b09a952e9a"
Jul 16 11:12:34.524533 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:34.524492    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"553fcf7f2f80943f4f8e666f32ffd779f73854c1bd611f32f628e0b09a952e9a"} err="failed to get container status \"553fcf7f2f80943f4f8e666f32ffd779f73854c1bd611f32f628e0b09a952e9a\": rpc error: code = NotFound desc = could not find container \"553fcf7f2f80943f4f8e666f32ffd779f73854c1bd611f32f628e0b09a952e9a\": container with ID starting with 553fcf7f2f80943f4f8e666f32ffd779f73854c1bd611f32f628e0b09a952e9a not found: ID does not exist"
Jul 16 11:12:34.537449 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:34.537425    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"]
Jul 16 11:12:34.542210 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:34.542189    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-307109/rate-test-model-307109-kserve-5c9dbbc49f-ljk25"]
Jul 16 11:12:35.802076 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:35.802044    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b" path="/var/lib/kubelet/pods/00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b/volumes"
Jul 16 11:12:43.892113 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:43.892075    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"]
Jul 16 11:12:43.892596 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:43.892311    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp" podUID="d7ee007e-ed40-4149-b728-8c9aacf0a9ef" containerName="maas-api" containerID="cri-o://9f96edf454ba20f3cbc4d121c49827272afdc0d449aa81dea027ea9ea85a0b5d" gracePeriod=30
Jul 16 11:12:44.336034 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.336004    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:12:44.448566 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.448466    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq26w\" (UniqueName: \"kubernetes.io/projected/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-kube-api-access-sq26w\") pod \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\" (UID: \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\") "
Jul 16 11:12:44.448566 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.448530    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-openshift-service-ca\") pod \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\" (UID: \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\") "
Jul 16 11:12:44.448791 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.448583    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-maas-api-tls\") pod \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\" (UID: \"d7ee007e-ed40-4149-b728-8c9aacf0a9ef\") "
Jul 16 11:12:44.448963 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.448935    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7ee007e-ed40-4149-b728-8c9aacf0a9ef" (UID: "d7ee007e-ed40-4149-b728-8c9aacf0a9ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:12:44.450607 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.450578    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7ee007e-ed40-4149-b728-8c9aacf0a9ef" (UID: "d7ee007e-ed40-4149-b728-8c9aacf0a9ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:12:44.450714 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.450621    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-kube-api-access-sq26w" (OuterVolumeSpecName: "kube-api-access-sq26w") pod "d7ee007e-ed40-4149-b728-8c9aacf0a9ef" (UID: "d7ee007e-ed40-4149-b728-8c9aacf0a9ef"). InnerVolumeSpecName "kube-api-access-sq26w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:12:44.549570 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.549534    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-maas-api-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:44.549570 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.549561    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq26w\" (UniqueName: \"kubernetes.io/projected/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-kube-api-access-sq26w\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:44.549570 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.549570    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7ee007e-ed40-4149-b728-8c9aacf0a9ef-openshift-service-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:44.567408 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.567382    2576 generic.go:356] "Generic (PLEG): container finished" podID="d7ee007e-ed40-4149-b728-8c9aacf0a9ef" containerID="9f96edf454ba20f3cbc4d121c49827272afdc0d449aa81dea027ea9ea85a0b5d" exitCode=0
Jul 16 11:12:44.567533 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.567426    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"
Jul 16 11:12:44.567533 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.567429    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp" event={"ID":"d7ee007e-ed40-4149-b728-8c9aacf0a9ef","Type":"ContainerDied","Data":"9f96edf454ba20f3cbc4d121c49827272afdc0d449aa81dea027ea9ea85a0b5d"}
Jul 16 11:12:44.567533 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.567489    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp" event={"ID":"d7ee007e-ed40-4149-b728-8c9aacf0a9ef","Type":"ContainerDied","Data":"b6db65b09d66dbfaba521de760fb74cb3e770ef876d15cf7226c4f64ae10c567"}
Jul 16 11:12:44.567533 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.567514    2576 scope.go:117] "RemoveContainer" containerID="9f96edf454ba20f3cbc4d121c49827272afdc0d449aa81dea027ea9ea85a0b5d"
Jul 16 11:12:44.576858 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.576839    2576 scope.go:117] "RemoveContainer" containerID="9f96edf454ba20f3cbc4d121c49827272afdc0d449aa81dea027ea9ea85a0b5d"
Jul 16 11:12:44.577112 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:12:44.577093    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f96edf454ba20f3cbc4d121c49827272afdc0d449aa81dea027ea9ea85a0b5d\": container with ID starting with 9f96edf454ba20f3cbc4d121c49827272afdc0d449aa81dea027ea9ea85a0b5d not found: ID does not exist" containerID="9f96edf454ba20f3cbc4d121c49827272afdc0d449aa81dea027ea9ea85a0b5d"
Jul 16 11:12:44.577161 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.577120    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f96edf454ba20f3cbc4d121c49827272afdc0d449aa81dea027ea9ea85a0b5d"} err="failed to get container status \"9f96edf454ba20f3cbc4d121c49827272afdc0d449aa81dea027ea9ea85a0b5d\": rpc error: code = NotFound desc = could not find container \"9f96edf454ba20f3cbc4d121c49827272afdc0d449aa81dea027ea9ea85a0b5d\": container with ID starting with 9f96edf454ba20f3cbc4d121c49827272afdc0d449aa81dea027ea9ea85a0b5d not found: ID does not exist"
Jul 16 11:12:44.589081 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.589059    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"]
Jul 16 11:12:44.595095 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:44.595075    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-046938-7884fc679f-55jsp"]
Jul 16 11:12:45.801108 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:45.801071    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7ee007e-ed40-4149-b728-8c9aacf0a9ef" path="/var/lib/kubelet/pods/d7ee007e-ed40-4149-b728-8c9aacf0a9ef/volumes"
Jul 16 11:12:57.742067 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:57.741975    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"]
Jul 16 11:12:57.742915 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:57.742849    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8" podUID="898d123c-1211-4746-95ed-606cb15efeb8" containerName="maas-api" containerID="cri-o://bd45bc14e232ea37cad57f5fe6708dadd9518aa82f3174a76d894e20bd83be81" gracePeriod=30
Jul 16 11:12:57.985545 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:57.985522    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:12:58.068882 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.068793    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/898d123c-1211-4746-95ed-606cb15efeb8-openshift-service-ca\") pod \"898d123c-1211-4746-95ed-606cb15efeb8\" (UID: \"898d123c-1211-4746-95ed-606cb15efeb8\") "
Jul 16 11:12:58.068882 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.068856    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/898d123c-1211-4746-95ed-606cb15efeb8-maas-api-tls\") pod \"898d123c-1211-4746-95ed-606cb15efeb8\" (UID: \"898d123c-1211-4746-95ed-606cb15efeb8\") "
Jul 16 11:12:58.069110 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.068888    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6khc\" (UniqueName: \"kubernetes.io/projected/898d123c-1211-4746-95ed-606cb15efeb8-kube-api-access-d6khc\") pod \"898d123c-1211-4746-95ed-606cb15efeb8\" (UID: \"898d123c-1211-4746-95ed-606cb15efeb8\") "
Jul 16 11:12:58.069221 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.069192    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/898d123c-1211-4746-95ed-606cb15efeb8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "898d123c-1211-4746-95ed-606cb15efeb8" (UID: "898d123c-1211-4746-95ed-606cb15efeb8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:12:58.070893 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.070866    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/898d123c-1211-4746-95ed-606cb15efeb8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "898d123c-1211-4746-95ed-606cb15efeb8" (UID: "898d123c-1211-4746-95ed-606cb15efeb8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:12:58.070982 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.070927    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/898d123c-1211-4746-95ed-606cb15efeb8-kube-api-access-d6khc" (OuterVolumeSpecName: "kube-api-access-d6khc") pod "898d123c-1211-4746-95ed-606cb15efeb8" (UID: "898d123c-1211-4746-95ed-606cb15efeb8"). InnerVolumeSpecName "kube-api-access-d6khc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:12:58.169567 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.169534    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/898d123c-1211-4746-95ed-606cb15efeb8-openshift-service-ca\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:58.169567 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.169560    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/898d123c-1211-4746-95ed-606cb15efeb8-maas-api-tls\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:58.169567 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.169570    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6khc\" (UniqueName: \"kubernetes.io/projected/898d123c-1211-4746-95ed-606cb15efeb8-kube-api-access-d6khc\") on node \"ip-10-0-128-213.ec2.internal\" DevicePath \"\""
Jul 16 11:12:58.631831 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.631793    2576 generic.go:356] "Generic (PLEG): container finished" podID="898d123c-1211-4746-95ed-606cb15efeb8" containerID="bd45bc14e232ea37cad57f5fe6708dadd9518aa82f3174a76d894e20bd83be81" exitCode=0
Jul 16 11:12:58.631831 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.631833    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"
Jul 16 11:12:58.632062 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.631872    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8" event={"ID":"898d123c-1211-4746-95ed-606cb15efeb8","Type":"ContainerDied","Data":"bd45bc14e232ea37cad57f5fe6708dadd9518aa82f3174a76d894e20bd83be81"}
Jul 16 11:12:58.632062 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.631921    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8" event={"ID":"898d123c-1211-4746-95ed-606cb15efeb8","Type":"ContainerDied","Data":"75f8601ef716c3e1bb1e9af7b1cc3734af91ac889cfc7b749e647510277f1870"}
Jul 16 11:12:58.632062 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.631942    2576 scope.go:117] "RemoveContainer" containerID="bd45bc14e232ea37cad57f5fe6708dadd9518aa82f3174a76d894e20bd83be81"
Jul 16 11:12:58.641222 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.641204    2576 scope.go:117] "RemoveContainer" containerID="bd45bc14e232ea37cad57f5fe6708dadd9518aa82f3174a76d894e20bd83be81"
Jul 16 11:12:58.641453 ip-10-0-128-213 kubenswrapper[2576]: E0716 11:12:58.641434    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd45bc14e232ea37cad57f5fe6708dadd9518aa82f3174a76d894e20bd83be81\": container with ID starting with bd45bc14e232ea37cad57f5fe6708dadd9518aa82f3174a76d894e20bd83be81 not found: ID does not exist" containerID="bd45bc14e232ea37cad57f5fe6708dadd9518aa82f3174a76d894e20bd83be81"
Jul 16 11:12:58.641502 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.641465    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd45bc14e232ea37cad57f5fe6708dadd9518aa82f3174a76d894e20bd83be81"} err="failed to get container status \"bd45bc14e232ea37cad57f5fe6708dadd9518aa82f3174a76d894e20bd83be81\": rpc error: code = NotFound desc = could not find container \"bd45bc14e232ea37cad57f5fe6708dadd9518aa82f3174a76d894e20bd83be81\": container with ID starting with bd45bc14e232ea37cad57f5fe6708dadd9518aa82f3174a76d894e20bd83be81 not found: ID does not exist"
Jul 16 11:12:58.654461 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.654436    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"]
Jul 16 11:12:58.657329 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:58.657307    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-307109-54dd44994-d6fp8"]
Jul 16 11:12:59.801190 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:12:59.801159    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="898d123c-1211-4746-95ed-606cb15efeb8" path="/var/lib/kubelet/pods/898d123c-1211-4746-95ed-606cb15efeb8/volumes"
Jul 16 11:13:13.574806 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:13.574772    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8f7d9f46-wff4j_2bb0e822-72d3-4622-8f58-aed1b5ee4add/copy-manifests/0.log"
Jul 16 11:13:13.583662 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:13.583638    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8f7d9f46-wff4j_2bb0e822-72d3-4622-8f58-aed1b5ee4add/manager/0.log"
Jul 16 11:13:16.227122 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:16.227086    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-98pr7_976132e9-6e42-4bf7-ae7f-f77ab99b3414/manager/0.log"
Jul 16 11:13:17.141921 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:17.141888    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58946d4b5c-pb828_f4db6a21-4d71-4e75-b7f7-c3e1979c1d88/kube-auth-proxy/0.log"
Jul 16 11:13:18.135864 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:18.135827    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-qsbwn_c601eca5-7ba4-4ab9-9d5f-d6e4f0f5dcc3/main/0.log"
Jul 16 11:13:18.489257 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:18.489160    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbct7n_5ba13897-88c8-48a9-a35a-d088eeeff396/main/0.log"
Jul 16 11:13:18.722685 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:18.722655    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-f749h_58a948c0-efda-4aab-a1bd-151f990f11ef/main/0.log"
Jul 16 11:13:27.190651 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:27.190618    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-r8z79_30449683-8c30-47cf-a574-474beccc52b0/global-pull-secret-syncer/0.log"
Jul 16 11:13:27.303526 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:27.303498    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lllwn_7f4f6727-e31d-4ec2-8dd0-91fcb059faf8/konnectivity-agent/0.log"
Jul 16 11:13:27.374841 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:27.374810    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-213.ec2.internal_09d3c0959bdc2c44d35e2b9b5a065fd9/haproxy/0.log"
Jul 16 11:13:32.754888 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:32.754856    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-98pr7_976132e9-6e42-4bf7-ae7f-f77ab99b3414/manager/0.log"
Jul 16 11:13:35.217831 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.217796    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-db5bc955b-67wg2_63191a58-2211-47a0-979c-f8f6821e540a/metrics-server/0.log"
Jul 16 11:13:35.524400 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.524312    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6n659_1c84770f-b939-48e8-a3fa-61f1d5fcccb5/node-exporter/0.log"
Jul 16 11:13:35.544912 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.544876    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6n659_1c84770f-b939-48e8-a3fa-61f1d5fcccb5/kube-rbac-proxy/0.log"
Jul 16 11:13:35.568083 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.568056    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6n659_1c84770f-b939-48e8-a3fa-61f1d5fcccb5/init-textfile/0.log"
Jul 16 11:13:35.712766 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.712716    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f076ad8d-df9e-4335-bf64-17174228fd5e/prometheus/0.log"
Jul 16 11:13:35.737502 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.737470    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f076ad8d-df9e-4335-bf64-17174228fd5e/config-reloader/0.log"
Jul 16 11:13:35.758153 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.758125    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f076ad8d-df9e-4335-bf64-17174228fd5e/thanos-sidecar/0.log"
Jul 16 11:13:35.783723 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.783648    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f076ad8d-df9e-4335-bf64-17174228fd5e/kube-rbac-proxy-web/0.log"
Jul 16 11:13:35.809035 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.809007    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f076ad8d-df9e-4335-bf64-17174228fd5e/kube-rbac-proxy/0.log"
Jul 16 11:13:35.836399 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.836374    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f076ad8d-df9e-4335-bf64-17174228fd5e/kube-rbac-proxy-thanos/0.log"
Jul 16 11:13:35.861146 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.861124    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f076ad8d-df9e-4335-bf64-17174228fd5e/init-config-reloader/0.log"
Jul 16 11:13:35.894291 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.894264    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-nngw4_bda89000-0ee6-41f4-a41e-631d65d020dc/prometheus-operator/0.log"
Jul 16 11:13:35.916246 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.916217    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-nngw4_bda89000-0ee6-41f4-a41e-631d65d020dc/kube-rbac-proxy/0.log"
Jul 16 11:13:35.945680 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:35.945658    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-pxxgl_f3d2592e-cef7-45c0-a246-e98739f3413a/prometheus-operator-admission-webhook/0.log"
Jul 16 11:13:36.081141 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:36.081057    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c9b88f75f-8l7tc_a6454579-1b08-4ca5-bd1a-94284b0d0ecb/thanos-query/0.log"
Jul 16 11:13:36.115903 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:36.115877    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c9b88f75f-8l7tc_a6454579-1b08-4ca5-bd1a-94284b0d0ecb/kube-rbac-proxy-web/0.log"
Jul 16 11:13:36.149280 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:36.149242    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c9b88f75f-8l7tc_a6454579-1b08-4ca5-bd1a-94284b0d0ecb/kube-rbac-proxy/0.log"
Jul 16 11:13:36.185568 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:36.185535    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c9b88f75f-8l7tc_a6454579-1b08-4ca5-bd1a-94284b0d0ecb/prom-label-proxy/0.log"
Jul 16 11:13:36.212415 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:36.212389    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c9b88f75f-8l7tc_a6454579-1b08-4ca5-bd1a-94284b0d0ecb/kube-rbac-proxy-rules/0.log"
Jul 16 11:13:36.241253 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:36.241216    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c9b88f75f-8l7tc_a6454579-1b08-4ca5-bd1a-94284b0d0ecb/kube-rbac-proxy-metrics/0.log"
Jul 16 11:13:37.204439 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.204403    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"]
Jul 16 11:13:37.204821 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.204807    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b" containerName="main"
Jul 16 11:13:37.204871 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.204823    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b" containerName="main"
Jul 16 11:13:37.204871 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.204834    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="898d123c-1211-4746-95ed-606cb15efeb8" containerName="maas-api"
Jul 16 11:13:37.204871 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.204839    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="898d123c-1211-4746-95ed-606cb15efeb8" containerName="maas-api"
Jul 16 11:13:37.204871 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.204854    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ee007e-ed40-4149-b728-8c9aacf0a9ef" containerName="maas-api"
Jul 16 11:13:37.204871 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.204860    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ee007e-ed40-4149-b728-8c9aacf0a9ef" containerName="maas-api"
Jul 16 11:13:37.204871 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.204868    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b" containerName="main"
Jul 16 11:13:37.204871 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.204874    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b" containerName="main"
Jul 16 11:13:37.205079 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.204946    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1b45c9c-cbf9-4f0a-8a99-3220f5100f6b" containerName="main"
Jul 16 11:13:37.205079 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.204953    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="898d123c-1211-4746-95ed-606cb15efeb8" containerName="maas-api"
Jul 16 11:13:37.205079 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.204959    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ee007e-ed40-4149-b728-8c9aacf0a9ef" containerName="maas-api"
Jul 16 11:13:37.205079 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.204968    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="00ab8772-c7e8-49fc-9eaa-84f6e6f4d82b" containerName="main"
Jul 16 11:13:37.208099 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.208079    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.210706 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.210677    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2z85x\"/\"openshift-service-ca.crt\""
Jul 16 11:13:37.210706 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.210694    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2z85x\"/\"kube-root-ca.crt\""
Jul 16 11:13:37.211384 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.211362    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2z85x\"/\"default-dockercfg-xjzjv\""
Jul 16 11:13:37.223708 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.223686    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"]
Jul 16 11:13:37.312469 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.312422    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/73b590a2-670f-431c-858f-1c5d286c3359-lib-modules\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.312469 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.312467    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73b590a2-670f-431c-858f-1c5d286c3359-sys\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.313008 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.312530    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/73b590a2-670f-431c-858f-1c5d286c3359-proc\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.313008 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.312592    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgkzb\" (UniqueName: \"kubernetes.io/projected/73b590a2-670f-431c-858f-1c5d286c3359-kube-api-access-kgkzb\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.313008 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.312628    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/73b590a2-670f-431c-858f-1c5d286c3359-podres\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.413303 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.413268    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/73b590a2-670f-431c-858f-1c5d286c3359-lib-modules\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.413303 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.413304    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73b590a2-670f-431c-858f-1c5d286c3359-sys\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.413483 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.413328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/73b590a2-670f-431c-858f-1c5d286c3359-proc\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.413483 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.413359    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgkzb\" (UniqueName: \"kubernetes.io/projected/73b590a2-670f-431c-858f-1c5d286c3359-kube-api-access-kgkzb\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.413483 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.413389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/73b590a2-670f-431c-858f-1c5d286c3359-podres\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.413483 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.413406    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73b590a2-670f-431c-858f-1c5d286c3359-sys\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.413483 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.413415    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/73b590a2-670f-431c-858f-1c5d286c3359-proc\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.413667 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.413415    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/73b590a2-670f-431c-858f-1c5d286c3359-lib-modules\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.413667 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.413510    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/73b590a2-670f-431c-858f-1c5d286c3359-podres\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.422385 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.422362    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgkzb\" (UniqueName: \"kubernetes.io/projected/73b590a2-670f-431c-858f-1c5d286c3359-kube-api-access-kgkzb\") pod \"perf-node-gather-daemonset-s4jkn\" (UID: \"73b590a2-670f-431c-858f-1c5d286c3359\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.473736 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.473656    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-wfnn6_c61920c1-33d8-43ae-b81e-be8e9b1cc13b/networking-console-plugin/0.log"
Jul 16 11:13:37.518430 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.518405    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.644905 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.644876    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"]
Jul 16 11:13:37.647287 ip-10-0-128-213 kubenswrapper[2576]: W0716 11:13:37.647261    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod73b590a2_670f_431c_858f_1c5d286c3359.slice/crio-432c746455165ea6e1b9bd27f78671b5d3c384478f16b816e26ff6bc61c9820f WatchSource:0}: Error finding container 432c746455165ea6e1b9bd27f78671b5d3c384478f16b816e26ff6bc61c9820f: Status 404 returned error can't find the container with id 432c746455165ea6e1b9bd27f78671b5d3c384478f16b816e26ff6bc61c9820f
Jul 16 11:13:37.808667 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.808631    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn" event={"ID":"73b590a2-670f-431c-858f-1c5d286c3359","Type":"ContainerStarted","Data":"4abce4f70b75a28198a2dc6ae682b84b01f1f8ac2e83b6f2736deb19e7bce208"}
Jul 16 11:13:37.808903 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.808675    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn" event={"ID":"73b590a2-670f-431c-858f-1c5d286c3359","Type":"ContainerStarted","Data":"432c746455165ea6e1b9bd27f78671b5d3c384478f16b816e26ff6bc61c9820f"}
Jul 16 11:13:37.808903 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.808776    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:37.829324 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:37.829274    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn" podStartSLOduration=0.829260972 podStartE2EDuration="829.260972ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:13:37.827321733 +0000 UTC m=+2472.564357016" watchObservedRunningTime="2026-07-16 11:13:37.829260972 +0000 UTC m=+2472.566296243"
Jul 16 11:13:40.390041 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:40.390013    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bbs64_fc917a38-428c-4044-bc83-21eb7e6b75ce/dns/0.log"
Jul 16 11:13:40.411943 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:40.411918    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bbs64_fc917a38-428c-4044-bc83-21eb7e6b75ce/kube-rbac-proxy/0.log"
Jul 16 11:13:40.485248 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:40.485216    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7l8m9_d3cc8c24-4def-40f0-9ee3-df7f26fdb61a/dns-node-resolver/0.log"
Jul 16 11:13:41.124158 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:41.124128    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-658996cf97-hss89_d61ca23e-84d5-4e18-82b6-bfd2fc874869/registry/0.log"
Jul 16 11:13:41.149272 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:41.149239    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d5zm6_2aeb2209-a85b-460e-a65c-cd36e86a5a8e/node-ca/0.log"
Jul 16 11:13:42.330233 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:42.330200    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58946d4b5c-pb828_f4db6a21-4d71-4e75-b7f7-c3e1979c1d88/kube-auth-proxy/0.log"
Jul 16 11:13:43.353058 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:43.353021    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bb4br_41674660-ce10-48c5-a5d8-a57669fa42a8/serve-healthcheck-canary/0.log"
Jul 16 11:13:43.823405 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:43.823334    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-s4jkn"
Jul 16 11:13:44.102004 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:44.101952    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z4vz5_2a5c84ae-5969-4224-b0e4-0afee0632152/kube-rbac-proxy/0.log"
Jul 16 11:13:44.131535 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:44.131509    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z4vz5_2a5c84ae-5969-4224-b0e4-0afee0632152/exporter/0.log"
Jul 16 11:13:44.158800 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:44.158766    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z4vz5_2a5c84ae-5969-4224-b0e4-0afee0632152/extractor/0.log"
Jul 16 11:13:47.691288 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:47.691213    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8f7d9f46-wff4j_2bb0e822-72d3-4622-8f58-aed1b5ee4add/manager/0.log"
Jul 16 11:13:47.701669 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:47.701646    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58946d4b5c-pb828_f4db6a21-4d71-4e75-b7f7-c3e1979c1d88/kube-auth-proxy/0.log"
Jul 16 11:13:47.733693 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:47.733659    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8f7d9f46-wff4j_2bb0e822-72d3-4622-8f58-aed1b5ee4add/copy-manifests/0.log"
Jul 16 11:13:49.222956 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:49.222929    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-85564789c6-8xbvg_aca3864e-af4d-455e-8ee1-dbfa3fc433d0/manager/0.log"
Jul 16 11:13:49.322651 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:49.322569    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58946d4b5c-pb828_f4db6a21-4d71-4e75-b7f7-c3e1979c1d88/kube-auth-proxy/0.log"
Jul 16 11:13:54.405381 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:54.405341    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-5dmzd_44bdc776-7b33-4be0-8ad7-827030065c04/migrator/0.log"
Jul 16 11:13:54.432717 ip-10-0-128-213 kubenswrapper[2576]: I0716 11:13:54.432686    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-5dmzd_44bdc776-7b33-4be0-8ad7-827030065c04/graceful-termination/0.log"