Jul 29 21:04:08.003205 ip-10-0-138-99 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 21:04:08.393264 ip-10-0-138-99 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 21:04:08.393264 ip-10-0-138-99 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 21:04:08.393264 ip-10-0-138-99 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 21:04:08.393264 ip-10-0-138-99 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 21:04:08.393264 ip-10-0-138-99 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 21:04:08.395134 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.395028    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 21:04:08.401585 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401560    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 21:04:08.401585 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401581    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 21:04:08.401585 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401585    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 21:04:08.401585 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401588    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401593    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401599    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401602    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401606    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401609    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401612    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401616    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401619    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401621    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401624    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401626    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401629    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401632    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401635    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401637    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401640    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401642    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401645    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401648    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 21:04:08.401756 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401650    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401653    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401662    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401665    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401668    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401670    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401674    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401677    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401679    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401682    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401685    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401688    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401690    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401693    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401696    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401699    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401702    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401704    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401707    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401709    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 21:04:08.402262 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401712    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401715    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401717    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401720    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401722    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401725    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401729    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401734    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401737    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401740    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401742    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401745    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401748    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401751    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401753    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401756    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401759    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401762    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401764    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401768    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 21:04:08.402761 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401770    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401773    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401776    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401779    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401781    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401784    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401788    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401791    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401794    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401796    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401799    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401802    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401804    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401807    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401809    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401812    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401814    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401816    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401819    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 21:04:08.403304 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401821    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401825    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401827    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401830    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401832    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401849    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401852    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401855    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401857    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401860    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401863    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401866    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401869    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401872    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401874    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401877    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 21:04:08.403759 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.401881    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 21:04:08.404468 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404454    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 21:04:08.404468 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404467    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404470    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404474    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404477    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404480    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404483    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404486    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404489    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404492    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404495    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404498    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404501    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404503    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404506    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404508    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404511    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404514    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404517    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404519    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404522    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 21:04:08.404524 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404525    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404528    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404531    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404533    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404536    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404539    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404541    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404544    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404546    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404549    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404552    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404555    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404557    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404560    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404563    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404565    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404567    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404570    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404572    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404575    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 21:04:08.405036 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404577    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404580    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404582    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404585    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404587    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404590    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404594    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404597    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404600    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404604    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404607    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404610    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404613    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404615    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404620    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404623    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404627    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404629    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 21:04:08.405525 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404632    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404634    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404637    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404640    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404642    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404644    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404647    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404650    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404653    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404655    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404659    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404662    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404664    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404666    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404669    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404671    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404674    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404676    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404679    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404683    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 21:04:08.405987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404686    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404689    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404692    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404695    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404698    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404701    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404704    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404707    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404710    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404713    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404715    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404718    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404720    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404723    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404725    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404728    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404731    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404733    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404736    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.404738    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 21:04:08.406490 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405190    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405197    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405200    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405204    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405207    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405210    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405212    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405215    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405220    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405223    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405226    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405229    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405232    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405234    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405237    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405240    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405243    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405246    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405249    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405252    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 21:04:08.406987 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405255    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405257    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405260    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405263    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405265    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405268    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405271    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405273    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405276    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405278    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405281    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405283    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405286    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405288    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405291    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405294    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405298    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405301    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405304    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 21:04:08.407486 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405308    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405310    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405313    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405316    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405319    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405321    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405326    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405328    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405334    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405337    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405340    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405344    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405346    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405349    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405351    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405354    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405356    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405359    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405361    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 21:04:08.407980 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405364    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405367    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405369    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405372    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405374    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405376    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405379    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405381    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405384    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405386    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405389    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405391    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405394    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405396    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405399    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405401    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405404    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405407    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405409    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405412    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 21:04:08.408478 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405414    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405418    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405420    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405424    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405430    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405433    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405436    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405439    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405441    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405444    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405446    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405449    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405452    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405454    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405457    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405459    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405462    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405464    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405467    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 21:04:08.408976 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405469    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.405473    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405548    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405564    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405570    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405574    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405578    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405582    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405586    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405590    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405593    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405596    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405600    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405603    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405607    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405610    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405613    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405616    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405623    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405626    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405633    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405636    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405639    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405643    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 21:04:08.409428 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405646    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405650    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405653    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405656    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405659    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405662    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405665    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405668    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405671    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405674    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405678    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405681    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405683    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405686    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405689    2580 flags.go:64] FLAG: --enable-server="true"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405692    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405698    2580 flags.go:64] FLAG: --event-burst="100"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405701    2580 flags.go:64] FLAG: --event-qps="50"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405704    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405707    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405710    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405713    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405717    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405719    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405722    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 29 21:04:08.410175 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405725    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405728    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405737    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405740    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405743    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405746    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405749    2580 flags.go:64] FLAG: --feature-gates=""
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405752    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405755    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405758    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405761    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405764    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405767    2580 flags.go:64] FLAG: --help="false"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405770    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405773    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405776    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405781    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405784    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405788    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405791    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405793    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405796    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405799    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 21:04:08.410789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405801    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405805    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405807    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405810    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405813    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405816    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405819    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405822    2580 flags.go:64] FLAG: --lock-file=""
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405825    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405827    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405830    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405847    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405859    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405862    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405865    2580 flags.go:64] FLAG: --logging-format="text"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405868    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405871    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405874    2580 flags.go:64] FLAG: --manifest-url=""
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405877    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405881    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405884    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405888    2580 flags.go:64] FLAG: --max-pods="110"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405891    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405894    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405897    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 21:04:08.411414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405902    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405905    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405908    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405911    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405918    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405921    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405924    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405927    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405930    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405935    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405938    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405941    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405944    2580 flags.go:64] FLAG: --port="10250"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405947    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405950    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0175d041ba2648785"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405953    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405956    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405959    2580 flags.go:64] FLAG: --register-node="true"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405962    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405965    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405974    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405977    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405980    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405984    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405987    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 21:04:08.412063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405990    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405993    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405996    2580 flags.go:64] FLAG: --runonce="false"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.405998    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406001    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406004    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406007    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406012    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406015    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406018    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406020    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406023    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406026    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406029    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406032    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406035    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406037    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406047    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406053    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406058    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406061    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406068    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406075    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406078    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406081    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 21:04:08.412737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406084    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406087    2580 flags.go:64] FLAG: --v="2"
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406091    2580 flags.go:64] FLAG: --version="false"
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406107    2580 flags.go:64] FLAG: --vmodule=""
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406115    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406119    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406255    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406260    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406263    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406267    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406270    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406272    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406275    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406277    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406281    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406284    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406286    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406289    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406291    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406294    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 21:04:08.413387 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406298    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406302    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406305    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406307    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406310    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406313    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406317    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406320    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406323    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406325    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406328    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406331    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406333    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406336    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406339    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406341    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406350    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406353    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406355    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406358    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 21:04:08.413899 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406360    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406363    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406365    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406368    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406371    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406373    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406379    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406382    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406385    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406387    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406390    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406393    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406396    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406400    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406403    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406406    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406409    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406411    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406415    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406418    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 21:04:08.414410 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406420    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406423    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406426    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406428    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406431    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406433    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406436    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406439    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406441    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406449    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406452    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406455    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406458    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406460    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406463    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406466    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406468    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406471    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406475    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406477    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 21:04:08.414932 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406480    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406483    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406486    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406488    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406490    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406493    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406496    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406499    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406501    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406505    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406509    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406511    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406514    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406516    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406519    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406522    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406524    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406527    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406530    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406532    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 21:04:08.415440 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406535    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406537    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406590    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406594    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.406597    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.406603    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.413765    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.413781    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.413802    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.413812    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413886    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413892    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413896    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413899    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 21:04:08.415953 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413902    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413905    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413907    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413910    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413913    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413916    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413918    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413921    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413924    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413927    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413930    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413932    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413935    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413937    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413940    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413943    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413946    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413949    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413952    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413954    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 21:04:08.416312 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413957    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413960    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413964    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413967    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413971    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413975    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413978    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413981    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413984    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413987    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413989    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413992    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413995    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.413997    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414000    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414002    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414005    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414008    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414010    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 21:04:08.416805 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414013    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414015    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414018    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414021    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414023    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414026    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414029    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414031    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414034    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414036    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414039    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414050    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414053    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414056    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414059    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414061    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414064    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414066    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414070    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414074    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 21:04:08.417310 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414076    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414079    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414082    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414084    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414087    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414090    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414092    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414095    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414097    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414100    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414103    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414105    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414108    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414110    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414113    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414115    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414118    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414121    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414123    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414126    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 21:04:08.417850 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414128    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414131    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414135    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414139    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414143    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414147    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414150    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414154    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414156    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414159    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414162    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414165    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414168    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414171    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414173    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 21:04:08.418373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414176    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.414181    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414291    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414295    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414299    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414302    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414305    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414308    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414311    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414314    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414317    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414319    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414322    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414325    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414328    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 21:04:08.418750 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414331    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414333    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414336    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414338    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414341    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414344    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414347    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414349    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414353    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414357    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414360    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414363    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414366    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414370    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414373    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414376    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414379    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414381    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414384    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 21:04:08.419158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414386    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414389    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414391    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414394    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414396    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414399    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414401    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414404    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414406    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414408    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414411    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414413    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414416    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414418    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414421    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414423    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414426    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414429    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414432    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414434    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 21:04:08.419640 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414436    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414439    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414442    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414444    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414447    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414449    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414452    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414454    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414457    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414460    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414462    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414465    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414467    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414470    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414472    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414475    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414478    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414480    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414482    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414485    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 21:04:08.420195 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414487    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414490    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414492    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414495    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414497    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414500    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414502    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414505    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414508    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414511    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414513    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414515    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414518    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414520    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414523    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414525    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414528    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414530    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414533    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414535    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 21:04:08.420674 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414538    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 21:04:08.421193 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414540    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 21:04:08.421193 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414543    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 21:04:08.421193 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414546    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 21:04:08.421193 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414548    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 21:04:08.421193 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414550    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 21:04:08.421193 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:08.414553    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 21:04:08.421193 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.414558    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 21:04:08.421193 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.414661    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 21:04:08.421193 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.416892    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 21:04:08.421193 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.418474    2580 server.go:1019] "Starting client certificate rotation"
Jul 29 21:04:08.421193 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.418584    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 21:04:08.421193 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.418650    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 21:04:08.421193 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.421026    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 21:04:08.427031 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.427004    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 21:04:08.440589 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.440571    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 29 21:04:08.442810 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.442793    2580 log.go:25] "Validated CRI v1 image API"
Jul 29 21:04:08.443616 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.443600    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 21:04:08.446254 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.446234    2580 fs.go:135] Filesystem UUIDs: map[105f2d88-ea59-4a9c-b25f-24f696f8b281:/dev/nvme0n1p4 136aa187-9434-4bb0-8887-465ac64a95c1:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 29 21:04:08.446329 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.446252    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 21:04:08.449967 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.449949    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 21:04:08.451309 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.451201    2580 manager.go:216] Machine: {Timestamp:2026-07-29 21:04:08.449883029 +0000 UTC m=+0.331157157 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107666 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b86a955382e3c2bc4d155164f21f9 SystemUUID:ec2b86a9-5538-2e3c-2bc4-d155164f21f9 BootID:c6e35fdc-f637-43a5-b893-e8edf7803f2c Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a5:85:83:0a:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a5:85:83:0a:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:f6:97:9f:43:c7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 21:04:08.451309 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.451301    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 21:04:08.451420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.451380    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 21:04:08.454483 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.454454    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 21:04:08.454618 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.454485    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-99.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 21:04:08.455878 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.455868    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 21:04:08.455912 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.455880    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 21:04:08.455912 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.455888    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 21:04:08.456605 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.456595    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 21:04:08.457510 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.457500    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 21:04:08.460263 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.460252    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 21:04:08.460411 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.460401    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 21:04:08.460443 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.460432    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 21:04:08.460470 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.460455    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 21:04:08.460516 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.460506    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 21:04:08.460548 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.460527    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 29 21:04:08.460548 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.460537    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 21:04:08.461124 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.461110    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 21:04:08.461182 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.461137    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 21:04:08.463323 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.463305    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 21:04:08.466295 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.466232    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 21:04:08.466445 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.466432    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 21:04:08.468365 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.468342    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 21:04:08.468365 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.468368    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 21:04:08.468470 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.468376    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 21:04:08.468470 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.468383    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 21:04:08.468470 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.468393    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 21:04:08.468470 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.468403    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 21:04:08.468470 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.468412    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 21:04:08.468470 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.468418    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 21:04:08.468470 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.468441    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 21:04:08.468470 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.468463    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 21:04:08.468470 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.468474    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 21:04:08.468694 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.468482    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 21:04:08.469649 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.469627    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 21:04:08.469726 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.469665    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 21:04:08.470763 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.470733    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 21:04:08.470763 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.470731    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 21:04:08.472919 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.472905    2580 server.go:1268] "Started kubelet"
Jul 29 21:04:08.473089 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.472994    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 21:04:08.473196 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.473040    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 21:04:08.473302 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.473287    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 21:04:08.473816 ip-10-0-138-99 systemd[1]: Started Kubernetes Kubelet.
Jul 29 21:04:08.474650 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.474476    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m6ts2"
Jul 29 21:04:08.474650 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.474488    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 21:04:08.475196 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.475178    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 21:04:08.479133 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.479116    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 21:04:08.480053 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.480036    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-99.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 21:04:08.481104 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.480077    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-99.ec2.internal.18c6de9868c15783  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-99.ec2.internal,UID:ip-10-0-138-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-99.ec2.internal,},FirstTimestamp:2026-07-29 21:04:08.472876931 +0000 UTC m=+0.354151059,LastTimestamp:2026-07-29 21:04:08.472876931 +0000 UTC m=+0.354151059,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-99.ec2.internal,}"
Jul 29 21:04:08.481918 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.481903    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 21:04:08.481973 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.481949    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 21:04:08.482020 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.481989    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found"
Jul 29 21:04:08.482749 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.482730    2580 factory.go:55] Registering systemd factory
Jul 29 21:04:08.482749 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.482753    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 29 21:04:08.482905 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.482774    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m6ts2"
Jul 29 21:04:08.482971 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.482957    2580 factory.go:153] Registering CRI-O factory
Jul 29 21:04:08.483022 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.482975    2580 factory.go:223] Registration of the crio container factory successfully
Jul 29 21:04:08.483070 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.483030    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 21:04:08.483070 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.483042    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 21:04:08.483070 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.483046    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 21:04:08.483070 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.483061    2580 factory.go:103] Registering Raw factory
Jul 29 21:04:08.483215 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.483077    2580 manager.go:1196] Started watching for new ooms in manager
Jul 29 21:04:08.483215 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.483189    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 21:04:08.483215 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.483198    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 21:04:08.483330 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.483312    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 21:04:08.483387 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.483377    2580 manager.go:349] Starting recovery of all containers
Jul 29 21:04:08.485195 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.485167    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 21:04:08.485319 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.485291    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 21:04:08.494048 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.494030    2580 manager.go:354] Recovery completed
Jul 29 21:04:08.500804 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.500788    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:04:08.502783 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.502759    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:04:08.502864 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.502796    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:04:08.502864 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.502808    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:04:08.503402 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.503388    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 21:04:08.503402 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.503401    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 21:04:08.503486 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.503418    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 21:04:08.503486 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.503433    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:04:08.504335 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.504311    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:04:08.504398 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.504349    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:04:08.504398 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.504367    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:04:08.506373 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.506360    2580 policy_none.go:49] "None policy: Start"
Jul 29 21:04:08.506424 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.506378    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 21:04:08.506424 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.506390    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 21:04:08.507704 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.507692    2580 policy_none.go:47] "Start"
Jul 29 21:04:08.543701 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.543557    2580 manager.go:337] "Starting Device Plugin manager"
Jul 29 21:04:08.543807 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.543716    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 21:04:08.543807 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.543728    2580 server.go:87] "Starting device plugin registration server"
Jul 29 21:04:08.544010 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.543998    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 21:04:08.544067 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.544011    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 21:04:08.544168 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.544150    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 21:04:08.544260 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.544246    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 21:04:08.544260 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.544260    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 21:04:08.544880 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.544846    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 21:04:08.544969 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.544918    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-99.ec2.internal\" not found"
Jul 29 21:04:08.645265 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.645178    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:04:08.646711 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.646251    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:04:08.646711 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.646293    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:04:08.646711 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.646305    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:04:08.646711 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.646336    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.655378 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.655355    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.655492 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.655389    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-99.ec2.internal\": node \"ip-10-0-138-99.ec2.internal\" not found"
Jul 29 21:04:08.671212 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.671186    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found"
Jul 29 21:04:08.707069 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.707041    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 21:04:08.708343 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.708325    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 21:04:08.708435 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.708362    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 21:04:08.708435 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.708382    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 21:04:08.708435 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.708416    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 21:04:08.710447 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.710427    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 21:04:08.773160 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.773122    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found"
Jul 29 21:04:08.810288 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.810241    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal"]
Jul 29 21:04:08.810390 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.810333    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:04:08.811518 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.811499    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:04:08.811616 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.811532    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:04:08.811616 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.811543    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:04:08.813332 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.813319    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:04:08.813487 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.813458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.813532 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.813503    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:04:08.814218 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.814200    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:04:08.814324 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.814233    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:04:08.814324 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.814239    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:04:08.814324 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.814249    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:04:08.814324 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.814264    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:04:08.814324 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.814279    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:04:08.815751 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.815733    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.815843 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.815763    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:04:08.816532 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.816512    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:04:08.816622 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.816550    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:04:08.816622 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.816562    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:04:08.839523 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.839501    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-99.ec2.internal\" not found" node="ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.843933 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.843914    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-99.ec2.internal\" not found" node="ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.873513 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.873486    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found"
Jul 29 21:04:08.885468 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.885447    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a6d41c68b970bb763324c8f000cd0b6e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-99.ec2.internal\" (UID: \"a6d41c68b970bb763324c8f000cd0b6e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.885534 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.885473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1008694150dc36b60aa75aea0060f22b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal\" (UID: \"1008694150dc36b60aa75aea0060f22b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.885534 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.885493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1008694150dc36b60aa75aea0060f22b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal\" (UID: \"1008694150dc36b60aa75aea0060f22b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.974089 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:08.974012    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found"
Jul 29 21:04:08.986334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.986308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a6d41c68b970bb763324c8f000cd0b6e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-99.ec2.internal\" (UID: \"a6d41c68b970bb763324c8f000cd0b6e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.986400 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.986333    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a6d41c68b970bb763324c8f000cd0b6e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-99.ec2.internal\" (UID: \"a6d41c68b970bb763324c8f000cd0b6e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.986400 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.986356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1008694150dc36b60aa75aea0060f22b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal\" (UID: \"1008694150dc36b60aa75aea0060f22b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.986400 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.986384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1008694150dc36b60aa75aea0060f22b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal\" (UID: \"1008694150dc36b60aa75aea0060f22b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.986496 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.986412    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1008694150dc36b60aa75aea0060f22b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal\" (UID: \"1008694150dc36b60aa75aea0060f22b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:08.986496 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:08.986432    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1008694150dc36b60aa75aea0060f22b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal\" (UID: \"1008694150dc36b60aa75aea0060f22b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:09.077082 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.077027    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found"
Jul 29 21:04:09.145796 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.145756    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:09.148063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.148044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:09.178198 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.178166    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found"
Jul 29 21:04:09.285151 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.285058    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found"
Jul 29 21:04:09.311342 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.311315    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 21:04:09.387923 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.387888    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:09.399571 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.399549    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 21:04:09.401624 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.401609    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:09.417430 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.417406    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 21:04:09.417554 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.417534    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 21:04:09.417606 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.417576    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 21:04:09.417656 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.417618    2580 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a078845a2198b4eb598323db375ed653-8c7fbedf04a78960.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": read tcp 10.0.138.99:58948->52.5.230.218:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal"
Jul 29 21:04:09.463201 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.463173    2580 apiserver.go:52] "Watching apiserver"
Jul 29 21:04:09.482464 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.482432    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 21:04:09.483275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.483254    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 21:04:09.484490 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.484470    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-phqpp","kube-system/konnectivity-agent-sxpkh","openshift-cluster-node-tuning-operator/tuned-8dzqz","openshift-image-registry/node-ca-25rxb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal","openshift-multus/multus-additional-cni-plugins-jk6th","openshift-multus/network-metrics-daemon-6m4sz","kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm","openshift-dns/node-resolver-f58qv","openshift-multus/multus-cxq9x","openshift-network-diagnostics/network-check-target-rgt2l","openshift-network-operator/iptables-alerter-glw9j","openshift-ovn-kubernetes/ovnkube-node-z4bvn"]
Jul 29 21:04:09.486247 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.486232    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:09.486315 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.486297    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:09.487479 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.487460    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sxpkh"
Jul 29 21:04:09.487608 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.487557    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 20:59:08 +0000 UTC" deadline="2028-04-23 08:47:24.755719436 +0000 UTC"
Jul 29 21:04:09.487674 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.487610    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15203h43m15.26811378s"
Jul 29 21:04:09.489043 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.489019    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.490429 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.490407    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hbbm9\""
Jul 29 21:04:09.490530 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.490440    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-25rxb"
Jul 29 21:04:09.493073 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.492128    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 21:04:09.493073 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.492631    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p2xbt\""
Jul 29 21:04:09.493073 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.492705    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 21:04:09.493269 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.493083    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 21:04:09.493269 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.493091    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 21:04:09.494526 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.494485    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.495393 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.495369    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 21:04:09.495512 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.495464    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ls92z\""
Jul 29 21:04:09.495512 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.495485    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 21:04:09.495828 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.495809    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 21:04:09.496206 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.496189    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 21:04:09.496452 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.496431    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:09.496532 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.496505    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:09.497799 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.497777    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 21:04:09.497935 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.497905    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.498393 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.498061    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 21:04:09.498393 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.498143    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 21:04:09.498393 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.498174    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 21:04:09.498393 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.498265    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 21:04:09.498393 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.498316    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s88d5\""
Jul 29 21:04:09.499608 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.499585    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f58qv"
Jul 29 21:04:09.500763 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.500747    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 21:04:09.500903 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.500764    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 21:04:09.500999 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.500928    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 21:04:09.501382 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.501365    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xd859\""
Jul 29 21:04:09.501597 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.501578    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.502610 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.502596    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 21:04:09.502685 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.502622    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-c9k4s\""
Jul 29 21:04:09.503252 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.503236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:09.503319 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.503295    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:09.503723 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.503701    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 21:04:09.504522 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.504508    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 21:04:09.504723 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.504708    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rv76t\""
Jul 29 21:04:09.504802 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.504786    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-glw9j"
Jul 29 21:04:09.506242 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.506225    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.508109 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.508087    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-85464\""
Jul 29 21:04:09.508216 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.508120    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 21:04:09.508216 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.508156    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 21:04:09.508216 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.508125    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 21:04:09.509656 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.509640    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 21:04:09.509749 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.509639    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5z9r9\""
Jul 29 21:04:09.510062 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.510041    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 21:04:09.510161 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.510074    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 21:04:09.510161 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.510048    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 21:04:09.510275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.510180    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 21:04:09.510275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.510193    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 21:04:09.523880 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.523860    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dhrds"
Jul 29 21:04:09.530150 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.530133    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 21:04:09.537157 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.535856    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dhrds"
Jul 29 21:04:09.584387 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.584349    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 21:04:09.588778 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.588752    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-log-socket\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.588947 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.588784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-registration-dir\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.588947 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.588805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-systemd\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.588947 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.588832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-os-release\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.588947 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.588874    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrr6x\" (UniqueName: \"kubernetes.io/projected/77711b35-c5f4-44b4-8db0-f739cdaad81b-kube-api-access-zrr6x\") pod \"iptables-alerter-glw9j\" (UID: \"77711b35-c5f4-44b4-8db0-f739cdaad81b\") " pod="openshift-network-operator/iptables-alerter-glw9j"
Jul 29 21:04:09.588947 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.588896    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-systemd-units\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.588952    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-ovnkube-script-lib\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.588989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a5822af-8e5a-4b6b-9797-38c1b32049b8-cni-binary-copy\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589035    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-socket-dir\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589052    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n28rz\" (UniqueName: \"kubernetes.io/projected/10c55495-ad3d-4ec5-a77c-2b639a74b4db-kube-api-access-n28rz\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589090    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/77711b35-c5f4-44b4-8db0-f739cdaad81b-iptables-alerter-script\") pod \"iptables-alerter-glw9j\" (UID: \"77711b35-c5f4-44b4-8db0-f739cdaad81b\") " pod="openshift-network-operator/iptables-alerter-glw9j"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589117    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7a5822af-8e5a-4b6b-9797-38c1b32049b8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589145    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-lib-modules\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-system-cni-dir\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589215    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-multus-cni-dir\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-run-k8s-cni-cncf-io\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-var-lib-cni-multus\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-run-systemd\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-run-ovn\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.589420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a5822af-8e5a-4b6b-9797-38c1b32049b8-cnibin\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589400    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-sys\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns9mz\" (UniqueName: \"kubernetes.io/projected/b2ce3236-8e63-46ab-9771-364aa885d23e-kube-api-access-ns9mz\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589449    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-var-lib-openvswitch\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589467    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-ovnkube-config\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk7cb\" (UniqueName: \"kubernetes.io/projected/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-kube-api-access-nk7cb\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxr7w\" (UniqueName: \"kubernetes.io/projected/7a5822af-8e5a-4b6b-9797-38c1b32049b8-kube-api-access-lxr7w\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589580    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-sys-fs\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-sysconfig\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-sysctl-conf\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589662    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-multus-socket-dir-parent\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589692    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-cni-netd\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-modprobe-d\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-kubernetes\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-tuned\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-cnibin\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.590352 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-etc-selinux\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589885    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/308b710d-1f57-4abb-9d06-273683523857-konnectivity-ca\") pod \"konnectivity-agent-sxpkh\" (UID: \"308b710d-1f57-4abb-9d06-273683523857\") " pod="kube-system/konnectivity-agent-sxpkh"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589905    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-var-lib-kubelet\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d44ed47a-d0fe-4536-a552-4ee051c80914-serviceca\") pod \"node-ca-25rxb\" (UID: \"d44ed47a-d0fe-4536-a552-4ee051c80914\") " pod="openshift-image-registry/node-ca-25rxb"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b3f9485e-51c3-4074-b7ed-aaf6aa491811-multus-daemon-config\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.589990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-device-dir\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-run-netns\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-run-multus-certs\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-dbus\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590187    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-run\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-host\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590243    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-hostroot\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5mrh\" (UniqueName: \"kubernetes.io/projected/b3f9485e-51c3-4074-b7ed-aaf6aa491811-kube-api-access-x5mrh\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590296    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-kubelet-config\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590333    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-cni-bin\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.591075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a5822af-8e5a-4b6b-9797-38c1b32049b8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590416    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2ce3236-8e63-46ab-9771-364aa885d23e-tmp\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590439    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwk9b\" (UniqueName: \"kubernetes.io/projected/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-kube-api-access-bwk9b\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590466    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/97bd114e-b39c-4454-be0a-a964b2db56f8-hosts-file\") pod \"node-resolver-f58qv\" (UID: \"97bd114e-b39c-4454-be0a-a964b2db56f8\") " pod="openshift-dns/node-resolver-f58qv"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97bd114e-b39c-4454-be0a-a964b2db56f8-tmp-dir\") pod \"node-resolver-f58qv\" (UID: \"97bd114e-b39c-4454-be0a-a964b2db56f8\") " pod="openshift-dns/node-resolver-f58qv"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590513    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-var-lib-cni-bin\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-kubelet\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-slash\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590611    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v7hf\" (UniqueName: \"kubernetes.io/projected/97bd114e-b39c-4454-be0a-a964b2db56f8-kube-api-access-7v7hf\") pod \"node-resolver-f58qv\" (UID: \"97bd114e-b39c-4454-be0a-a964b2db56f8\") " pod="openshift-dns/node-resolver-f58qv"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn6z6\" (UniqueName: \"kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6\") pod \"network-check-target-rgt2l\" (UID: \"b23d3306-5c16-4423-b2e0-28eec2615c61\") " pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-multus-conf-dir\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-node-log\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590733    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-run-netns\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-etc-openvswitch\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590771    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-env-overrides\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590789    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/77711b35-c5f4-44b4-8db0-f739cdaad81b-host-slash\") pod \"iptables-alerter-glw9j\" (UID: \"77711b35-c5f4-44b4-8db0-f739cdaad81b\") " pod="openshift-network-operator/iptables-alerter-glw9j"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/308b710d-1f57-4abb-9d06-273683523857-agent-certs\") pod \"konnectivity-agent-sxpkh\" (UID: \"308b710d-1f57-4abb-9d06-273683523857\") " pod="kube-system/konnectivity-agent-sxpkh"
Jul 29 21:04:09.591680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590839    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-sysctl-d\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.592368 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d44ed47a-d0fe-4536-a552-4ee051c80914-host\") pod \"node-ca-25rxb\" (UID: \"d44ed47a-d0fe-4536-a552-4ee051c80914\") " pod="openshift-image-registry/node-ca-25rxb"
Jul 29 21:04:09.592368 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-run-ovn-kubernetes\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.592368 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590926    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a5822af-8e5a-4b6b-9797-38c1b32049b8-system-cni-dir\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.592368 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7a5822af-8e5a-4b6b-9797-38c1b32049b8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.592368 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.590984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vww5\" (UniqueName: \"kubernetes.io/projected/d44ed47a-d0fe-4536-a552-4ee051c80914-kube-api-access-5vww5\") pod \"node-ca-25rxb\" (UID: \"d44ed47a-d0fe-4536-a552-4ee051c80914\") " pod="openshift-image-registry/node-ca-25rxb"
Jul 29 21:04:09.592368 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.591008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-var-lib-kubelet\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.592368 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.591049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-run-openvswitch\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.592368 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.591077    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-ovn-node-metrics-cert\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.592368 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.591117    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a5822af-8e5a-4b6b-9797-38c1b32049b8-os-release\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.592368 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.591153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3f9485e-51c3-4074-b7ed-aaf6aa491811-cni-binary-copy\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.592368 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.591170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-etc-kubernetes\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.691908 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.691882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-slash\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.692063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.691922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v7hf\" (UniqueName: \"kubernetes.io/projected/97bd114e-b39c-4454-be0a-a964b2db56f8-kube-api-access-7v7hf\") pod \"node-resolver-f58qv\" (UID: \"97bd114e-b39c-4454-be0a-a964b2db56f8\") " pod="openshift-dns/node-resolver-f58qv"
Jul 29 21:04:09.692063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.691953    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn6z6\" (UniqueName: \"kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6\") pod \"network-check-target-rgt2l\" (UID: \"b23d3306-5c16-4423-b2e0-28eec2615c61\") " pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:09.692063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.691993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-multus-conf-dir\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.692063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692023    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-multus-conf-dir\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.692252 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.691995    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-slash\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.692403 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-node-log\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.692469 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-run-netns\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.692527 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-etc-openvswitch\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.692527 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-env-overrides\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.692629 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/77711b35-c5f4-44b4-8db0-f739cdaad81b-host-slash\") pod \"iptables-alerter-glw9j\" (UID: \"77711b35-c5f4-44b4-8db0-f739cdaad81b\") " pod="openshift-network-operator/iptables-alerter-glw9j"
Jul 29 21:04:09.692629 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-node-log\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.692629 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/308b710d-1f57-4abb-9d06-273683523857-agent-certs\") pod \"konnectivity-agent-sxpkh\" (UID: \"308b710d-1f57-4abb-9d06-273683523857\") " pod="kube-system/konnectivity-agent-sxpkh"
Jul 29 21:04:09.692629 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-etc-openvswitch\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.692834 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692646    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-run-netns\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.692834 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/77711b35-c5f4-44b4-8db0-f739cdaad81b-host-slash\") pod \"iptables-alerter-glw9j\" (UID: \"77711b35-c5f4-44b4-8db0-f739cdaad81b\") " pod="openshift-network-operator/iptables-alerter-glw9j"
Jul 29 21:04:09.692834 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692702    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-sysctl-d\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.692834 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d44ed47a-d0fe-4536-a552-4ee051c80914-host\") pod \"node-ca-25rxb\" (UID: \"d44ed47a-d0fe-4536-a552-4ee051c80914\") " pod="openshift-image-registry/node-ca-25rxb"
Jul 29 21:04:09.692834 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-run-ovn-kubernetes\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.692834 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a5822af-8e5a-4b6b-9797-38c1b32049b8-system-cni-dir\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.693186 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.692842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7a5822af-8e5a-4b6b-9797-38c1b32049b8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.693186 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.693064    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 21:04:09.694123 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.693837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-env-overrides\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.696328 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.696302    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vww5\" (UniqueName: \"kubernetes.io/projected/d44ed47a-d0fe-4536-a552-4ee051c80914-kube-api-access-5vww5\") pod \"node-ca-25rxb\" (UID: \"d44ed47a-d0fe-4536-a552-4ee051c80914\") " pod="openshift-image-registry/node-ca-25rxb"
Jul 29 21:04:09.696439 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.696355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-var-lib-kubelet\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.696439 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.696389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-run-openvswitch\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.696439 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.696418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-ovn-node-metrics-cert\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.696589 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.696449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a5822af-8e5a-4b6b-9797-38c1b32049b8-os-release\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.696589 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.696480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3f9485e-51c3-4074-b7ed-aaf6aa491811-cni-binary-copy\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.696589 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.696509    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-etc-kubernetes\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.696707 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.696588    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-run-openvswitch\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.697133 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697107    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d44ed47a-d0fe-4536-a552-4ee051c80914-host\") pod \"node-ca-25rxb\" (UID: \"d44ed47a-d0fe-4536-a552-4ee051c80914\") " pod="openshift-image-registry/node-ca-25rxb"
Jul 29 21:04:09.697258 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-var-lib-kubelet\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.697374 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697336    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-etc-kubernetes\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.697374 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a5822af-8e5a-4b6b-9797-38c1b32049b8-os-release\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.697555 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697469    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-run-ovn-kubernetes\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697798    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-log-socket\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-sysctl-d\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697843    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a5822af-8e5a-4b6b-9797-38c1b32049b8-system-cni-dir\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-registration-dir\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-systemd\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-os-release\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697960    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-registration-dir\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697976    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrr6x\" (UniqueName: \"kubernetes.io/projected/77711b35-c5f4-44b4-8db0-f739cdaad81b-kube-api-access-zrr6x\") pod \"iptables-alerter-glw9j\" (UID: \"77711b35-c5f4-44b4-8db0-f739cdaad81b\") " pod="openshift-network-operator/iptables-alerter-glw9j"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-systemd-units\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698012    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-systemd\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698034    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-ovnkube-script-lib\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a5822af-8e5a-4b6b-9797-38c1b32049b8-cni-binary-copy\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-socket-dir\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n28rz\" (UniqueName: \"kubernetes.io/projected/10c55495-ad3d-4ec5-a77c-2b639a74b4db-kube-api-access-n28rz\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/77711b35-c5f4-44b4-8db0-f739cdaad81b-iptables-alerter-script\") pod \"iptables-alerter-glw9j\" (UID: \"77711b35-c5f4-44b4-8db0-f739cdaad81b\") " pod="openshift-network-operator/iptables-alerter-glw9j"
Jul 29 21:04:09.701249 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698271    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-os-release\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7a5822af-8e5a-4b6b-9797-38c1b32049b8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698327    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3f9485e-51c3-4074-b7ed-aaf6aa491811-cni-binary-copy\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.697909    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/308b710d-1f57-4abb-9d06-273683523857-agent-certs\") pod \"konnectivity-agent-sxpkh\" (UID: \"308b710d-1f57-4abb-9d06-273683523857\") " pod="kube-system/konnectivity-agent-sxpkh"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-lib-modules\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698446    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-system-cni-dir\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-lib-modules\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-multus-cni-dir\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-run-k8s-cni-cncf-io\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698525    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-systemd-units\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698564    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-var-lib-cni-multus\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7a5822af-8e5a-4b6b-9797-38c1b32049b8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-run-systemd\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698589    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-var-lib-cni-multus\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-run-ovn\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698653    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-run-ovn\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698694    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-run-systemd\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.702275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698715    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-socket-dir\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698787    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-run-k8s-cni-cncf-io\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a5822af-8e5a-4b6b-9797-38c1b32049b8-cnibin\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-sys\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698901    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-system-cni-dir\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698945    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-multus-cni-dir\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698953    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns9mz\" (UniqueName: \"kubernetes.io/projected/b2ce3236-8e63-46ab-9771-364aa885d23e-kube-api-access-ns9mz\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698989    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a5822af-8e5a-4b6b-9797-38c1b32049b8-cnibin\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/77711b35-c5f4-44b4-8db0-f739cdaad81b-iptables-alerter-script\") pod \"iptables-alerter-glw9j\" (UID: \"77711b35-c5f4-44b4-8db0-f739cdaad81b\") " pod="openshift-network-operator/iptables-alerter-glw9j"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.698877    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7a5822af-8e5a-4b6b-9797-38c1b32049b8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699027    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-ovnkube-script-lib\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-var-lib-openvswitch\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699044    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-sys\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-ovnkube-config\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699087    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.703192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-var-lib-openvswitch\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699197    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-log-socket\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk7cb\" (UniqueName: \"kubernetes.io/projected/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-kube-api-access-nk7cb\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699223    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a5822af-8e5a-4b6b-9797-38c1b32049b8-cni-binary-copy\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699248    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxr7w\" (UniqueName: \"kubernetes.io/projected/7a5822af-8e5a-4b6b-9797-38c1b32049b8-kube-api-access-lxr7w\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-sys-fs\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-sysconfig\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-sysctl-conf\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-multus-socket-dir-parent\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699375    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-sys-fs\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-cni-netd\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-cni-netd\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-modprobe-d\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699450    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-kubernetes\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-multus-socket-dir-parent\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699527    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-ovnkube-config\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699471    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-sysconfig\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.704140 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-kubernetes\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-tuned\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-cnibin\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-modprobe-d\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-etc-selinux\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/308b710d-1f57-4abb-9d06-273683523857-konnectivity-ca\") pod \"konnectivity-agent-sxpkh\" (UID: \"308b710d-1f57-4abb-9d06-273683523857\") " pod="kube-system/konnectivity-agent-sxpkh"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-var-lib-kubelet\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699713    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-sysctl-conf\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d44ed47a-d0fe-4536-a552-4ee051c80914-serviceca\") pod \"node-ca-25rxb\" (UID: \"d44ed47a-d0fe-4536-a552-4ee051c80914\") " pod="openshift-image-registry/node-ca-25rxb"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699734    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-etc-selinux\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b3f9485e-51c3-4074-b7ed-aaf6aa491811-multus-daemon-config\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699795    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-cnibin\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-device-dir\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-run-netns\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-run-multus-certs\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.699866    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-dbus\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:09.704758 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.699887    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.699901    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zn6z6 for pod openshift-network-diagnostics/network-check-target-rgt2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699920    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-run\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699942    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-host\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.699964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-hostroot\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.700003    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6 podName:b23d3306-5c16-4423-b2e0-28eec2615c61 nodeName:}" failed. No retries permitted until 2026-07-29 21:04:10.199978018 +0000 UTC m=+2.081252156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zn6z6" (UniqueName: "kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6") pod "network-check-target-rgt2l" (UID: "b23d3306-5c16-4423-b2e0-28eec2615c61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700013    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-hostroot\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700025    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5mrh\" (UniqueName: \"kubernetes.io/projected/b3f9485e-51c3-4074-b7ed-aaf6aa491811-kube-api-access-x5mrh\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-kubelet-config\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-cni-bin\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700135    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a5822af-8e5a-4b6b-9797-38c1b32049b8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2ce3236-8e63-46ab-9771-364aa885d23e-tmp\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwk9b\" (UniqueName: \"kubernetes.io/projected/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-kube-api-access-bwk9b\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/97bd114e-b39c-4454-be0a-a964b2db56f8-hosts-file\") pod \"node-resolver-f58qv\" (UID: \"97bd114e-b39c-4454-be0a-a964b2db56f8\") " pod="openshift-dns/node-resolver-f58qv"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700254    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d44ed47a-d0fe-4536-a552-4ee051c80914-serviceca\") pod \"node-ca-25rxb\" (UID: \"d44ed47a-d0fe-4536-a552-4ee051c80914\") " pod="openshift-image-registry/node-ca-25rxb"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97bd114e-b39c-4454-be0a-a964b2db56f8-tmp-dir\") pod \"node-resolver-f58qv\" (UID: \"97bd114e-b39c-4454-be0a-a964b2db56f8\") " pod="openshift-dns/node-resolver-f58qv"
Jul 29 21:04:09.705334 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700289    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-var-lib-cni-bin\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-kubelet\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700348    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-var-lib-kubelet\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700391    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-kubelet\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700441    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-run-multus-certs\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b3f9485e-51c3-4074-b7ed-aaf6aa491811-multus-daemon-config\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10c55495-ad3d-4ec5-a77c-2b639a74b4db-device-dir\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-dbus\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.700583    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-run\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700633    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b2ce3236-8e63-46ab-9771-364aa885d23e-host\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.700657    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret podName:f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3 nodeName:}" failed. No retries permitted until 2026-07-29 21:04:10.200636704 +0000 UTC m=+2.081910824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret") pod "global-pull-secret-syncer-phqpp" (UID: "f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700688    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-run-netns\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700753    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-host-cni-bin\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700796    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-kubelet-config\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.700867    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:09.700936    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs podName:ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b nodeName:}" failed. No retries permitted until 2026-07-29 21:04:10.200903264 +0000 UTC m=+2.082177386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs") pod "network-metrics-daemon-6m4sz" (UID: "ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:09.705854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700937    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a5822af-8e5a-4b6b-9797-38c1b32049b8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.706695 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.700980    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/97bd114e-b39c-4454-be0a-a964b2db56f8-hosts-file\") pod \"node-resolver-f58qv\" (UID: \"97bd114e-b39c-4454-be0a-a964b2db56f8\") " pod="openshift-dns/node-resolver-f58qv"
Jul 29 21:04:09.706695 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.701054    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3f9485e-51c3-4074-b7ed-aaf6aa491811-host-var-lib-cni-bin\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.706695 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.701253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97bd114e-b39c-4454-be0a-a964b2db56f8-tmp-dir\") pod \"node-resolver-f58qv\" (UID: \"97bd114e-b39c-4454-be0a-a964b2db56f8\") " pod="openshift-dns/node-resolver-f58qv"
Jul 29 21:04:09.706695 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.701359    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-ovn-node-metrics-cert\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.706695 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.701444    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/308b710d-1f57-4abb-9d06-273683523857-konnectivity-ca\") pod \"konnectivity-agent-sxpkh\" (UID: \"308b710d-1f57-4abb-9d06-273683523857\") " pod="kube-system/konnectivity-agent-sxpkh"
Jul 29 21:04:09.706695 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.702431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v7hf\" (UniqueName: \"kubernetes.io/projected/97bd114e-b39c-4454-be0a-a964b2db56f8-kube-api-access-7v7hf\") pod \"node-resolver-f58qv\" (UID: \"97bd114e-b39c-4454-be0a-a964b2db56f8\") " pod="openshift-dns/node-resolver-f58qv"
Jul 29 21:04:09.706695 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.702758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b2ce3236-8e63-46ab-9771-364aa885d23e-etc-tuned\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.706695 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.704453    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2ce3236-8e63-46ab-9771-364aa885d23e-tmp\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.707534 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:09.707490    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d41c68b970bb763324c8f000cd0b6e.slice/crio-77a155f1b3afbdb04b382bd937169330645db85165095d06c271ac42a44bce43 WatchSource:0}: Error finding container 77a155f1b3afbdb04b382bd937169330645db85165095d06c271ac42a44bce43: Status 404 returned error can't find the container with id 77a155f1b3afbdb04b382bd937169330645db85165095d06c271ac42a44bce43
Jul 29 21:04:09.708542 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:09.708516    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1008694150dc36b60aa75aea0060f22b.slice/crio-34327e14a69e567303ab58193de542110b01bd9c9bc07b26ddd47a517f7e4e39 WatchSource:0}: Error finding container 34327e14a69e567303ab58193de542110b01bd9c9bc07b26ddd47a517f7e4e39: Status 404 returned error can't find the container with id 34327e14a69e567303ab58193de542110b01bd9c9bc07b26ddd47a517f7e4e39
Jul 29 21:04:09.709769 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.709740    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrr6x\" (UniqueName: \"kubernetes.io/projected/77711b35-c5f4-44b4-8db0-f739cdaad81b-kube-api-access-zrr6x\") pod \"iptables-alerter-glw9j\" (UID: \"77711b35-c5f4-44b4-8db0-f739cdaad81b\") " pod="openshift-network-operator/iptables-alerter-glw9j"
Jul 29 21:04:09.709978 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.709955    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vww5\" (UniqueName: \"kubernetes.io/projected/d44ed47a-d0fe-4536-a552-4ee051c80914-kube-api-access-5vww5\") pod \"node-ca-25rxb\" (UID: \"d44ed47a-d0fe-4536-a552-4ee051c80914\") " pod="openshift-image-registry/node-ca-25rxb"
Jul 29 21:04:09.710307 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.710280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n28rz\" (UniqueName: \"kubernetes.io/projected/10c55495-ad3d-4ec5-a77c-2b639a74b4db-kube-api-access-n28rz\") pod \"aws-ebs-csi-driver-node-v52pm\" (UID: \"10c55495-ad3d-4ec5-a77c-2b639a74b4db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.711327 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.711301    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk7cb\" (UniqueName: \"kubernetes.io/projected/ea9e9d47-2e06-42bb-a685-e0e07f9d121c-kube-api-access-nk7cb\") pod \"ovnkube-node-z4bvn\" (UID: \"ea9e9d47-2e06-42bb-a685-e0e07f9d121c\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.711526 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.711508    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwk9b\" (UniqueName: \"kubernetes.io/projected/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-kube-api-access-bwk9b\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:09.711904 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.711836    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns9mz\" (UniqueName: \"kubernetes.io/projected/b2ce3236-8e63-46ab-9771-364aa885d23e-kube-api-access-ns9mz\") pod \"tuned-8dzqz\" (UID: \"b2ce3236-8e63-46ab-9771-364aa885d23e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.712066 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.712035    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxr7w\" (UniqueName: \"kubernetes.io/projected/7a5822af-8e5a-4b6b-9797-38c1b32049b8-kube-api-access-lxr7w\") pod \"multus-additional-cni-plugins-jk6th\" (UID: \"7a5822af-8e5a-4b6b-9797-38c1b32049b8\") " pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.712190 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.712174    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5mrh\" (UniqueName: \"kubernetes.io/projected/b3f9485e-51c3-4074-b7ed-aaf6aa491811-kube-api-access-x5mrh\") pod \"multus-cxq9x\" (UID: \"b3f9485e-51c3-4074-b7ed-aaf6aa491811\") " pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.714508 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.714493    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:04:09.815755 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.815663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sxpkh"
Jul 29 21:04:09.825110 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:09.825086    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308b710d_1f57_4abb_9d06_273683523857.slice/crio-4b29d63a7fa8775c5b3421437755509d9f2caa551cee1662abbe34a05d1dc9af WatchSource:0}: Error finding container 4b29d63a7fa8775c5b3421437755509d9f2caa551cee1662abbe34a05d1dc9af: Status 404 returned error can't find the container with id 4b29d63a7fa8775c5b3421437755509d9f2caa551cee1662abbe34a05d1dc9af
Jul 29 21:04:09.834644 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.834624    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8dzqz"
Jul 29 21:04:09.843085 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:09.843064    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2ce3236_8e63_46ab_9771_364aa885d23e.slice/crio-ba613be6e0e184ab469add7c74035f4e1dbf652ba1641f6c9fbdf07e38b15334 WatchSource:0}: Error finding container ba613be6e0e184ab469add7c74035f4e1dbf652ba1641f6c9fbdf07e38b15334: Status 404 returned error can't find the container with id ba613be6e0e184ab469add7c74035f4e1dbf652ba1641f6c9fbdf07e38b15334
Jul 29 21:04:09.850544 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.850521    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-25rxb"
Jul 29 21:04:09.859158 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:09.859125    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44ed47a_d0fe_4536_a552_4ee051c80914.slice/crio-8ed8429c24c7df654ee8deed7ea2432c7c48c81b3ce9071124ad97378e9d0385 WatchSource:0}: Error finding container 8ed8429c24c7df654ee8deed7ea2432c7c48c81b3ce9071124ad97378e9d0385: Status 404 returned error can't find the container with id 8ed8429c24c7df654ee8deed7ea2432c7c48c81b3ce9071124ad97378e9d0385
Jul 29 21:04:09.864602 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.864580    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jk6th"
Jul 29 21:04:09.871395 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.871375    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm"
Jul 29 21:04:09.873491 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:09.873467    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a5822af_8e5a_4b6b_9797_38c1b32049b8.slice/crio-92b9e36cc422accd821151d09ffafd5d686e9f604456ce0b0b12790c5111fd24 WatchSource:0}: Error finding container 92b9e36cc422accd821151d09ffafd5d686e9f604456ce0b0b12790c5111fd24: Status 404 returned error can't find the container with id 92b9e36cc422accd821151d09ffafd5d686e9f604456ce0b0b12790c5111fd24
Jul 29 21:04:09.879055 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:09.879033    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c55495_ad3d_4ec5_a77c_2b639a74b4db.slice/crio-e39381d763afec864dd6def3eb5d465c16c728dcf83a1725ffc145011edbdd7d WatchSource:0}: Error finding container e39381d763afec864dd6def3eb5d465c16c728dcf83a1725ffc145011edbdd7d: Status 404 returned error can't find the container with id e39381d763afec864dd6def3eb5d465c16c728dcf83a1725ffc145011edbdd7d
Jul 29 21:04:09.887173 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.887154    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f58qv"
Jul 29 21:04:09.905458 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.905439    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 21:04:09.910242 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.910225    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cxq9x"
Jul 29 21:04:09.918181 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:09.918158    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3f9485e_51c3_4074_b7ed_aaf6aa491811.slice/crio-52f3b0e8d3c22780b65543479978ecb43bfb4db13294c619819f294341476ab6 WatchSource:0}: Error finding container 52f3b0e8d3c22780b65543479978ecb43bfb4db13294c619819f294341476ab6: Status 404 returned error can't find the container with id 52f3b0e8d3c22780b65543479978ecb43bfb4db13294c619819f294341476ab6
Jul 29 21:04:09.933749 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.933723    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-glw9j"
Jul 29 21:04:09.940004 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:09.939984    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:09.941282 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:09.941259    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77711b35_c5f4_44b4_8db0_f739cdaad81b.slice/crio-4f1ef076d29d395387276ef3aa6375ce8ab00e92370bcdc14b4d6b1502afd6b2 WatchSource:0}: Error finding container 4f1ef076d29d395387276ef3aa6375ce8ab00e92370bcdc14b4d6b1502afd6b2: Status 404 returned error can't find the container with id 4f1ef076d29d395387276ef3aa6375ce8ab00e92370bcdc14b4d6b1502afd6b2
Jul 29 21:04:09.948648 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:09.948616    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea9e9d47_2e06_42bb_a685_e0e07f9d121c.slice/crio-89184e605325780023bd6dfb96edc48dffcec2235f442cec5ecdbf2db727724f WatchSource:0}: Error finding container 89184e605325780023bd6dfb96edc48dffcec2235f442cec5ecdbf2db727724f: Status 404 returned error can't find the container with id 89184e605325780023bd6dfb96edc48dffcec2235f442cec5ecdbf2db727724f
Jul 29 21:04:10.204252 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.204177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn6z6\" (UniqueName: \"kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6\") pod \"network-check-target-rgt2l\" (UID: \"b23d3306-5c16-4423-b2e0-28eec2615c61\") " pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:10.204252 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.204237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:10.204485 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.204271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:10.204485 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:10.204373    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:04:10.204485 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:10.204378    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:10.204485 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:10.204387    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:10.204485 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:10.204437    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs podName:ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b nodeName:}" failed. No retries permitted until 2026-07-29 21:04:11.204419877 +0000 UTC m=+3.085694013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs") pod "network-metrics-daemon-6m4sz" (UID: "ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:10.204485 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:10.204395    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:04:10.204485 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:10.204454    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret podName:f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3 nodeName:}" failed. No retries permitted until 2026-07-29 21:04:11.204446131 +0000 UTC m=+3.085720249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret") pod "global-pull-secret-syncer-phqpp" (UID: "f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:10.204485 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:10.204459    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zn6z6 for pod openshift-network-diagnostics/network-check-target-rgt2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:10.204905 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:10.204506    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6 podName:b23d3306-5c16-4423-b2e0-28eec2615c61 nodeName:}" failed. No retries permitted until 2026-07-29 21:04:11.204489677 +0000 UTC m=+3.085763813 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zn6z6" (UniqueName: "kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6") pod "network-check-target-rgt2l" (UID: "b23d3306-5c16-4423-b2e0-28eec2615c61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:10.539251 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.539015    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 20:59:09 +0000 UTC" deadline="2028-03-12 00:05:54.467524528 +0000 UTC"
Jul 29 21:04:10.539251 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.539127    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14187h1m43.928402977s"
Jul 29 21:04:10.710086 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.710049    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:10.710283 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:10.710209    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:10.710543 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.710494    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:10.710651 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:10.710628    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:10.727064 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.726669    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-25rxb" event={"ID":"d44ed47a-d0fe-4536-a552-4ee051c80914","Type":"ContainerStarted","Data":"8ed8429c24c7df654ee8deed7ea2432c7c48c81b3ce9071124ad97378e9d0385"}
Jul 29 21:04:10.740993 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.740955    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn" event={"ID":"ea9e9d47-2e06-42bb-a685-e0e07f9d121c","Type":"ContainerStarted","Data":"89184e605325780023bd6dfb96edc48dffcec2235f442cec5ecdbf2db727724f"}
Jul 29 21:04:10.745152 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.745115    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-glw9j" event={"ID":"77711b35-c5f4-44b4-8db0-f739cdaad81b","Type":"ContainerStarted","Data":"4f1ef076d29d395387276ef3aa6375ce8ab00e92370bcdc14b4d6b1502afd6b2"}
Jul 29 21:04:10.751019 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.750987    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 21:04:10.758538 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.758479    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cxq9x" event={"ID":"b3f9485e-51c3-4074-b7ed-aaf6aa491811","Type":"ContainerStarted","Data":"52f3b0e8d3c22780b65543479978ecb43bfb4db13294c619819f294341476ab6"}
Jul 29 21:04:10.765333 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.765219    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8dzqz" event={"ID":"b2ce3236-8e63-46ab-9771-364aa885d23e","Type":"ContainerStarted","Data":"ba613be6e0e184ab469add7c74035f4e1dbf652ba1641f6c9fbdf07e38b15334"}
Jul 29 21:04:10.782386 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.782324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sxpkh" event={"ID":"308b710d-1f57-4abb-9d06-273683523857","Type":"ContainerStarted","Data":"4b29d63a7fa8775c5b3421437755509d9f2caa551cee1662abbe34a05d1dc9af"}
Jul 29 21:04:10.801415 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.801266    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" event={"ID":"1008694150dc36b60aa75aea0060f22b","Type":"ContainerStarted","Data":"34327e14a69e567303ab58193de542110b01bd9c9bc07b26ddd47a517f7e4e39"}
Jul 29 21:04:10.809792 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.809755    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal" event={"ID":"a6d41c68b970bb763324c8f000cd0b6e","Type":"ContainerStarted","Data":"77a155f1b3afbdb04b382bd937169330645db85165095d06c271ac42a44bce43"}
Jul 29 21:04:10.819697 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.819663    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f58qv" event={"ID":"97bd114e-b39c-4454-be0a-a964b2db56f8","Type":"ContainerStarted","Data":"c68ca2a4002d04496b17630bfc867fa70bcb1b1a4126e7900af502484f8e8d60"}
Jul 29 21:04:10.836546 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.836496    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm" event={"ID":"10c55495-ad3d-4ec5-a77c-2b639a74b4db","Type":"ContainerStarted","Data":"e39381d763afec864dd6def3eb5d465c16c728dcf83a1725ffc145011edbdd7d"}
Jul 29 21:04:10.848263 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:10.848227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jk6th" event={"ID":"7a5822af-8e5a-4b6b-9797-38c1b32049b8","Type":"ContainerStarted","Data":"92b9e36cc422accd821151d09ffafd5d686e9f604456ce0b0b12790c5111fd24"}
Jul 29 21:04:11.214524 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:11.213368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn6z6\" (UniqueName: \"kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6\") pod \"network-check-target-rgt2l\" (UID: \"b23d3306-5c16-4423-b2e0-28eec2615c61\") " pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:11.214524 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:11.213470    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:11.214524 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:11.213507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:11.214524 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:11.213658    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:11.214524 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:11.213729    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs podName:ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b nodeName:}" failed. No retries permitted until 2026-07-29 21:04:13.213707778 +0000 UTC m=+5.094981896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs") pod "network-metrics-daemon-6m4sz" (UID: "ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:11.214524 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:11.214221    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:04:11.214524 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:11.214244    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:04:11.214524 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:11.214259    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zn6z6 for pod openshift-network-diagnostics/network-check-target-rgt2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:11.214524 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:11.214314    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6 podName:b23d3306-5c16-4423-b2e0-28eec2615c61 nodeName:}" failed. No retries permitted until 2026-07-29 21:04:13.214296732 +0000 UTC m=+5.095570871 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zn6z6" (UniqueName: "kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6") pod "network-check-target-rgt2l" (UID: "b23d3306-5c16-4423-b2e0-28eec2615c61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:11.214524 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:11.214391    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:11.214524 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:11.214425    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret podName:f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3 nodeName:}" failed. No retries permitted until 2026-07-29 21:04:13.214413779 +0000 UTC m=+5.095687914 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret") pod "global-pull-secret-syncer-phqpp" (UID: "f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:11.432961 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:11.432909    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 21:04:11.540127 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:11.540025    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 20:59:09 +0000 UTC" deadline="2028-04-15 06:31:03.910027724 +0000 UTC"
Jul 29 21:04:11.540127 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:11.540074    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15009h26m52.369959585s"
Jul 29 21:04:11.709448 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:11.709410    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:11.709664 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:11.709576    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:12.712643 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:12.712004    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:12.712643 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:12.712152    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:12.712643 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:12.712230    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:12.712643 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:12.712307    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:13.232632 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:13.232580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:13.232826 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:13.232646    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:13.232826 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:13.232683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn6z6\" (UniqueName: \"kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6\") pod \"network-check-target-rgt2l\" (UID: \"b23d3306-5c16-4423-b2e0-28eec2615c61\") " pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:13.233002 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:13.232937    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:04:13.233002 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:13.232970    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:04:13.233002 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:13.232986    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zn6z6 for pod openshift-network-diagnostics/network-check-target-rgt2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:13.233141 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:13.233023    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:13.233141 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:13.233057    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6 podName:b23d3306-5c16-4423-b2e0-28eec2615c61 nodeName:}" failed. No retries permitted until 2026-07-29 21:04:17.233034459 +0000 UTC m=+9.114308602 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zn6z6" (UniqueName: "kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6") pod "network-check-target-rgt2l" (UID: "b23d3306-5c16-4423-b2e0-28eec2615c61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:13.233141 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:13.233098    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret podName:f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3 nodeName:}" failed. No retries permitted until 2026-07-29 21:04:17.233080082 +0000 UTC m=+9.114354204 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret") pod "global-pull-secret-syncer-phqpp" (UID: "f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:13.233302 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:13.233175    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:13.233302 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:13.233212    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs podName:ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b nodeName:}" failed. No retries permitted until 2026-07-29 21:04:17.233199758 +0000 UTC m=+9.114473881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs") pod "network-metrics-daemon-6m4sz" (UID: "ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:13.709031 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:13.708945    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:13.709207 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:13.709099    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:14.708979 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:14.708944    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:14.709425 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:14.708944    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:14.709425 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:14.709156    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:14.709425 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:14.709172    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:15.709162 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:15.709124    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:15.709666 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:15.709273    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:16.711246 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:16.711066    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:16.711246 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:16.711223    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:16.711752 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:16.711618    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:16.711752 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:16.711710    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:17.269732 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:17.269691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn6z6\" (UniqueName: \"kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6\") pod \"network-check-target-rgt2l\" (UID: \"b23d3306-5c16-4423-b2e0-28eec2615c61\") " pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:17.269953 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:17.269785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:17.269953 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:17.269817    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:17.270086 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:17.269978    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:17.270086 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:17.269995    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:04:17.270086 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:17.270019    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:04:17.270086 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:17.270033    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zn6z6 for pod openshift-network-diagnostics/network-check-target-rgt2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:17.270086 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:17.269979    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:17.270086 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:17.270053    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs podName:ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b nodeName:}" failed. No retries permitted until 2026-07-29 21:04:25.270033551 +0000 UTC m=+17.151307669 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs") pod "network-metrics-daemon-6m4sz" (UID: "ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:17.270392 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:17.270104    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6 podName:b23d3306-5c16-4423-b2e0-28eec2615c61 nodeName:}" failed. No retries permitted until 2026-07-29 21:04:25.270087348 +0000 UTC m=+17.151361469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zn6z6" (UniqueName: "kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6") pod "network-check-target-rgt2l" (UID: "b23d3306-5c16-4423-b2e0-28eec2615c61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:17.270392 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:17.270121    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret podName:f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3 nodeName:}" failed. No retries permitted until 2026-07-29 21:04:25.270111425 +0000 UTC m=+17.151385546 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret") pod "global-pull-secret-syncer-phqpp" (UID: "f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:17.708894 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:17.708801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:17.709055 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:17.708972    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:18.710704 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:18.710669    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:18.711164 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:18.710784    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:18.712507 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:18.712314    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:18.712507 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:18.712457    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:19.709231 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:19.709198    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:19.709425 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:19.709343    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:20.709632 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:20.709549    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:20.710101 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:20.709661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:20.710101 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:20.709749    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:20.710101 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:20.709729    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:21.708982 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:21.708951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:21.709177 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:21.709115    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:22.709440 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:22.709401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:22.709440 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:22.709442    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:22.709900 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:22.709546    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:22.709900 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:22.709680    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:23.709471 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:23.709435    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:23.709952 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:23.709589    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:24.708680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:24.708613    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:24.708891 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:24.708764    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:24.708891 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:24.708814    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:24.709016 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:24.708947    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:25.330994 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:25.330957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:25.330994 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:25.330998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:25.331538 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:25.331028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn6z6\" (UniqueName: \"kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6\") pod \"network-check-target-rgt2l\" (UID: \"b23d3306-5c16-4423-b2e0-28eec2615c61\") " pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:25.331538 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:25.331146    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:25.331538 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:25.331176    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:04:25.331538 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:25.331196    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:04:25.331538 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:25.331205    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:25.331538 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:25.331219    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs podName:ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b nodeName:}" failed. No retries permitted until 2026-07-29 21:04:41.331203058 +0000 UTC m=+33.212477178 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs") pod "network-metrics-daemon-6m4sz" (UID: "ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:25.331538 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:25.331253    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret podName:f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3 nodeName:}" failed. No retries permitted until 2026-07-29 21:04:41.331240495 +0000 UTC m=+33.212514611 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret") pod "global-pull-secret-syncer-phqpp" (UID: "f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:25.331538 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:25.331208    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zn6z6 for pod openshift-network-diagnostics/network-check-target-rgt2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:25.331538 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:25.331316    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6 podName:b23d3306-5c16-4423-b2e0-28eec2615c61 nodeName:}" failed. No retries permitted until 2026-07-29 21:04:41.331306201 +0000 UTC m=+33.212580316 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zn6z6" (UniqueName: "kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6") pod "network-check-target-rgt2l" (UID: "b23d3306-5c16-4423-b2e0-28eec2615c61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:25.709407 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:25.709369    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:25.709600 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:25.709525    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:26.708701 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:26.708658    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:26.709251 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:26.708801    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:26.709251 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:26.708859    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:26.709251 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:26.708970    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:27.709057 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:27.709020    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:27.709544 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:27.709167    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:28.709993 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:28.709950    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:28.710440 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:28.710083    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:28.710440 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:28.710159    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:28.710440 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:28.710274    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:29.709077 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.708926    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:29.709180 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:29.709160    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:29.900832 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.900548    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:04:29.904548 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.904518    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea9e9d47-2e06-42bb-a685-e0e07f9d121c" containerID="cf501728342d0e4614536affa8f8e349015c523d7740efa0ed6682a5827d75c8" exitCode=1
Jul 29 21:04:29.904637 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.904603    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn" event={"ID":"ea9e9d47-2e06-42bb-a685-e0e07f9d121c","Type":"ContainerStarted","Data":"1f2711da07d49f7e01ebd2892c695ece56e97a97426671cf9c088792855ed19b"}
Jul 29 21:04:29.904703 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.904683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn" event={"ID":"ea9e9d47-2e06-42bb-a685-e0e07f9d121c","Type":"ContainerStarted","Data":"5d13b26fddef4bc416293e7b7f0461e0c5333c9974658318a34d61a67f1be8bc"}
Jul 29 21:04:29.904760 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.904735    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn" event={"ID":"ea9e9d47-2e06-42bb-a685-e0e07f9d121c","Type":"ContainerStarted","Data":"7f26ead746b2654b6415ea3a70fd563cf2d3312e834f73f94e3ad83ae96dd9be"}
Jul 29 21:04:29.904816 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.904785    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn" event={"ID":"ea9e9d47-2e06-42bb-a685-e0e07f9d121c","Type":"ContainerDied","Data":"cf501728342d0e4614536affa8f8e349015c523d7740efa0ed6682a5827d75c8"}
Jul 29 21:04:29.904905 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.904857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn" event={"ID":"ea9e9d47-2e06-42bb-a685-e0e07f9d121c","Type":"ContainerStarted","Data":"4caa627a3edc56adfd288282f729b43d171f2cb71964e8c6e6fc9c539979b999"}
Jul 29 21:04:29.910821 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.908576    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cxq9x" event={"ID":"b3f9485e-51c3-4074-b7ed-aaf6aa491811","Type":"ContainerStarted","Data":"26af7c316d1c17b89f59038d0347b8c44921c98d894655ca366187520e8499ac"}
Jul 29 21:04:29.916093 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.916063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8dzqz" event={"ID":"b2ce3236-8e63-46ab-9771-364aa885d23e","Type":"ContainerStarted","Data":"f2e4d08d9cf31ca2c1847bcde0de29b60a632f62b7d39d130ad92b64bbf370d6"}
Jul 29 21:04:29.918658 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.918472    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal" event={"ID":"a6d41c68b970bb763324c8f000cd0b6e","Type":"ContainerStarted","Data":"e7f37c332515704ca5ac3947e4e19c6315b01f76a70886bf2c0f34eb83615da1"}
Jul 29 21:04:29.930322 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.930258    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cxq9x" podStartSLOduration=1.062832832 podStartE2EDuration="20.930243781s" totalImagesPullingTime="19.867410949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:29.929553551 +0000 UTC m=+21.810827915" watchObservedRunningTime="2026-07-29 21:04:29.930243781 +0000 UTC m=+21.811517919"
Jul 29 21:04:29.951709 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.951436    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8dzqz" podStartSLOduration=1.414550942 podStartE2EDuration="20.951415641s" totalImagesPullingTime="19.536864699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:29.951027246 +0000 UTC m=+21.832301382" watchObservedRunningTime="2026-07-29 21:04:29.951415641 +0000 UTC m=+21.832689781"
Jul 29 21:04:29.965702 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:29.965649    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal" podStartSLOduration=20.965629135 podStartE2EDuration="20.965629135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:29.965001313 +0000 UTC m=+21.846275474" watchObservedRunningTime="2026-07-29 21:04:29.965629135 +0000 UTC m=+21.846903274"
Jul 29 21:04:30.708928 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.708894    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:30.708928 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.708925    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:30.709145 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:30.709018    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:30.709205 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:30.709182    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:30.922322 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.922245    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sxpkh" event={"ID":"308b710d-1f57-4abb-9d06-273683523857","Type":"ContainerStarted","Data":"85466ccbe6a29f07961cea6e970923b2a178713a7fbe47ba1f0dc2bb3d8edf09"}
Jul 29 21:04:30.923852 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.923682    2580 generic.go:356] "Generic (PLEG): container finished" podID="1008694150dc36b60aa75aea0060f22b" containerID="d9de1d531883a494e7562ba78648d280ee711ab1664db9322fcb55719fe1d730" exitCode=0
Jul 29 21:04:30.923946 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.923760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" event={"ID":"1008694150dc36b60aa75aea0060f22b","Type":"ContainerDied","Data":"d9de1d531883a494e7562ba78648d280ee711ab1664db9322fcb55719fe1d730"}
Jul 29 21:04:30.925576 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.925545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f58qv" event={"ID":"97bd114e-b39c-4454-be0a-a964b2db56f8","Type":"ContainerStarted","Data":"daf6bc2a7cd37eaab6436044a30eb583c3a8d8123789ebf78fb2f0a46cedbe51"}
Jul 29 21:04:30.928457 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.928434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm" event={"ID":"10c55495-ad3d-4ec5-a77c-2b639a74b4db","Type":"ContainerStarted","Data":"aa23c13f9f845fa006754bfb97b8c12c8b209de4936dd4ca37e25257d85ea665"}
Jul 29 21:04:30.929715 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.929693    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a5822af-8e5a-4b6b-9797-38c1b32049b8" containerID="5c703b65a5904f0e13869c023f71cdd49f6f086c1e8873a4431b35502b2906ec" exitCode=0
Jul 29 21:04:30.929813 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.929760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jk6th" event={"ID":"7a5822af-8e5a-4b6b-9797-38c1b32049b8","Type":"ContainerDied","Data":"5c703b65a5904f0e13869c023f71cdd49f6f086c1e8873a4431b35502b2906ec"}
Jul 29 21:04:30.931142 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.931124    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-25rxb" event={"ID":"d44ed47a-d0fe-4536-a552-4ee051c80914","Type":"ContainerStarted","Data":"67a1962cb3eebd4450c87f17c67cb05c5c0a6d32960a700b91be7099da70d8b3"}
Jul 29 21:04:30.933938 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.933922    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:04:30.934286 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.934269    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn" event={"ID":"ea9e9d47-2e06-42bb-a685-e0e07f9d121c","Type":"ContainerStarted","Data":"fdccd275322da650b4a46c100bbecb386f6dc7c9222c422be22bd48200975d5d"}
Jul 29 21:04:30.935612 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.935587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-glw9j" event={"ID":"77711b35-c5f4-44b4-8db0-f739cdaad81b","Type":"ContainerStarted","Data":"29cee0aa444cc0cce5e31dee3ce56aec787baf41549a114e18c4c9e39ac24f9d"}
Jul 29 21:04:30.938592 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.938556    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sxpkh" podStartSLOduration=10.95482978 podStartE2EDuration="21.938544952s" totalImagesPullingTime="10.983715172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:30.938146191 +0000 UTC m=+22.819420331" watchObservedRunningTime="2026-07-29 21:04:30.938544952 +0000 UTC m=+22.819819089"
Jul 29 21:04:30.970515 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:30.970466    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-25rxb" podStartSLOduration=10.351404075 podStartE2EDuration="21.970447653s" totalImagesPullingTime="11.619043578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:30.954227893 +0000 UTC m=+22.835502030" watchObservedRunningTime="2026-07-29 21:04:30.970447653 +0000 UTC m=+22.851721792"
Jul 29 21:04:31.026047 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:31.026005    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-f58qv" podStartSLOduration=2.567331293 podStartE2EDuration="22.025990238s" totalImagesPullingTime="19.458658945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:31.00929571 +0000 UTC m=+22.890569844" watchObservedRunningTime="2026-07-29 21:04:31.025990238 +0000 UTC m=+22.907264375"
Jul 29 21:04:31.026171 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:31.026154    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-glw9j" podStartSLOduration=2.774034882 podStartE2EDuration="22.026149771s" totalImagesPullingTime="19.252114889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:31.025869553 +0000 UTC m=+22.907143706" watchObservedRunningTime="2026-07-29 21:04:31.026149771 +0000 UTC m=+22.907423909"
Jul 29 21:04:31.174412 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:31.174359    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 21:04:31.559779 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:31.559501    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T21:04:31.174376806Z","UUID":"9578efac-081d-4c0b-b2a3-ff48986eac6f","Handler":null,"Name":"","Endpoint":""}
Jul 29 21:04:31.561509 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:31.561483    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 21:04:31.561650 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:31.561525    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 21:04:31.708731 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:31.708693    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:31.708919 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:31.708868    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:31.940397 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:31.940362    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm" event={"ID":"10c55495-ad3d-4ec5-a77c-2b639a74b4db","Type":"ContainerStarted","Data":"9e264a03a8dce250943dd8b97c0912188b757b7bc0778ee0e663f51d9859f8be"}
Jul 29 21:04:31.942269 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:31.942229    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" event={"ID":"1008694150dc36b60aa75aea0060f22b","Type":"ContainerStarted","Data":"7f929d184d2fee9b2217c556486187f79939bd5549180ea64675b3c7a64eae59"}
Jul 29 21:04:32.708951 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:32.708915    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:32.709152 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:32.708915    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:32.709152 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:32.709062    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:32.709152 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:32.709105    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:32.946561 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:32.946524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm" event={"ID":"10c55495-ad3d-4ec5-a77c-2b639a74b4db","Type":"ContainerStarted","Data":"9408ed8d6f389b3587400a5e6cb67a7f95e722eb9af5c125f18d96807948d5ee"}
Jul 29 21:04:32.950049 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:32.950028    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:04:32.950566 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:32.950505    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn" event={"ID":"ea9e9d47-2e06-42bb-a685-e0e07f9d121c","Type":"ContainerStarted","Data":"cd3bbcb0bb6e90ee21e810d14e9d55899630247065a648d3a3bcae0d60952956"}
Jul 29 21:04:32.964302 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:32.964259    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" podStartSLOduration=23.964243092 podStartE2EDuration="23.964243092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:31.958038525 +0000 UTC m=+23.839312663" watchObservedRunningTime="2026-07-29 21:04:32.964243092 +0000 UTC m=+24.845517231"
Jul 29 21:04:32.964500 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:32.964468    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v52pm" podStartSLOduration=2.100484315 podStartE2EDuration="23.96445981s" totalImagesPullingTime="21.863975495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:32.964053793 +0000 UTC m=+24.845327953" watchObservedRunningTime="2026-07-29 21:04:32.96445981 +0000 UTC m=+24.845733976"
Jul 29 21:04:33.709498 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:33.709462    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:33.709702 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:33.709632    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:34.708686 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:34.708651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:34.708686 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:34.708688    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:34.709145 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:34.708788    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:34.709145 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:34.708933    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:34.712116 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:34.712094    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sxpkh"
Jul 29 21:04:34.712246 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:34.712172    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sxpkh"
Jul 29 21:04:34.712419 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:34.712402    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sxpkh"
Jul 29 21:04:34.712811 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:34.712795    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sxpkh"
Jul 29 21:04:35.708811 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:35.708637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:35.709411 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:35.708900    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:35.965832 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:35.965751    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a5822af-8e5a-4b6b-9797-38c1b32049b8" containerID="8164eb02c9bb845c9029344dc76ef58cbd8ce8927d1c5d719375a3f109ca9210" exitCode=0
Jul 29 21:04:35.966098 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:35.965831    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jk6th" event={"ID":"7a5822af-8e5a-4b6b-9797-38c1b32049b8","Type":"ContainerDied","Data":"8164eb02c9bb845c9029344dc76ef58cbd8ce8927d1c5d719375a3f109ca9210"}
Jul 29 21:04:35.969484 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:35.969465    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:04:35.969869 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:35.969831    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn" event={"ID":"ea9e9d47-2e06-42bb-a685-e0e07f9d121c","Type":"ContainerStarted","Data":"d17881bb6fb1152c30e6d38a76de999748b175443970c1a4913825590eb0bdca"}
Jul 29 21:04:35.970088 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:35.970065    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:35.970204 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:35.970137    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:35.970297 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:35.970279    2580 scope.go:117] "RemoveContainer" containerID="cf501728342d0e4614536affa8f8e349015c523d7740efa0ed6682a5827d75c8"
Jul 29 21:04:35.985532 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:35.985514    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:35.986051 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:35.986034    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:36.708805 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:36.708775    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:36.709041 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:36.708914    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:36.709041 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:36.708970    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:36.709568 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:36.709115    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:36.975272 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:36.975209    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:04:36.975571 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:36.975547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn" event={"ID":"ea9e9d47-2e06-42bb-a685-e0e07f9d121c","Type":"ContainerStarted","Data":"08d96017e56cfc0e373c301908f5e24be305b9456a88160eb9b05936f680689f"}
Jul 29 21:04:36.975656 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:36.975637    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 29 21:04:37.005478 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:37.005431    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn" podStartSLOduration=8.542566765 podStartE2EDuration="28.005415447s" totalImagesPullingTime="19.462848682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:37.004692072 +0000 UTC m=+28.885966230" watchObservedRunningTime="2026-07-29 21:04:37.005415447 +0000 UTC m=+28.886689584"
Jul 29 21:04:37.363183 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:37.362880    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6m4sz"]
Jul 29 21:04:37.363354 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:37.363223    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:37.363354 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:37.363332    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:37.367236 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:37.367211    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-phqpp"]
Jul 29 21:04:37.367402 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:37.367313    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:37.367463 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:37.367412    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:37.367997 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:37.367975    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rgt2l"]
Jul 29 21:04:37.368090 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:37.368075    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:37.368190 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:37.368170    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:37.979751 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:37.979716    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a5822af-8e5a-4b6b-9797-38c1b32049b8" containerID="7a40959c325ee845780d0dac7cba29946af581bdd29c68e588680ec90cbac60d" exitCode=0
Jul 29 21:04:37.980146 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:37.979805    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jk6th" event={"ID":"7a5822af-8e5a-4b6b-9797-38c1b32049b8","Type":"ContainerDied","Data":"7a40959c325ee845780d0dac7cba29946af581bdd29c68e588680ec90cbac60d"}
Jul 29 21:04:37.980146 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:37.979888    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 29 21:04:38.063415 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:38.063384    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:04:38.710247 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:38.710176    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:38.710384 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:38.710258    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:38.710384 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:38.710343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:38.710464 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:38.710436    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:39.708715 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:39.708680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:39.709267 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:39.708807    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phqpp" podUID="f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3"
Jul 29 21:04:40.712168 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:40.712136    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:40.712545 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:40.712237    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:04:40.712545 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:40.712290    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:40.712545 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:40.712368    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rgt2l" podUID="b23d3306-5c16-4423-b2e0-28eec2615c61"
Jul 29 21:04:40.989191 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:40.989109    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a5822af-8e5a-4b6b-9797-38c1b32049b8" containerID="d873971fec1dd25715f3c68dce783ee879e8f85263e833826126368f51c20101" exitCode=0
Jul 29 21:04:40.989191 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:40.989172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jk6th" event={"ID":"7a5822af-8e5a-4b6b-9797-38c1b32049b8","Type":"ContainerDied","Data":"d873971fec1dd25715f3c68dce783ee879e8f85263e833826126368f51c20101"}
Jul 29 21:04:41.352062 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.351984    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:41.352062 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.352021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:41.352062 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.352043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn6z6\" (UniqueName: \"kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6\") pod \"network-check-target-rgt2l\" (UID: \"b23d3306-5c16-4423-b2e0-28eec2615c61\") " pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:41.352266 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:41.352149    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:41.352266 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:41.352163    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:04:41.352266 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:41.352177    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:04:41.352266 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:41.352185    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zn6z6 for pod openshift-network-diagnostics/network-check-target-rgt2l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:41.352266 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:41.352149    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:41.352266 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:41.352216    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret podName:f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3 nodeName:}" failed. No retries permitted until 2026-07-29 21:05:13.35220059 +0000 UTC m=+65.233474706 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret") pod "global-pull-secret-syncer-phqpp" (UID: "f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:04:41.352266 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:41.352235    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6 podName:b23d3306-5c16-4423-b2e0-28eec2615c61 nodeName:}" failed. No retries permitted until 2026-07-29 21:05:13.352225806 +0000 UTC m=+65.233499922 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zn6z6" (UniqueName: "kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6") pod "network-check-target-rgt2l" (UID: "b23d3306-5c16-4423-b2e0-28eec2615c61") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:04:41.352266 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:41.352250    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs podName:ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b nodeName:}" failed. No retries permitted until 2026-07-29 21:05:13.352242483 +0000 UTC m=+65.233516820 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs") pod "network-metrics-daemon-6m4sz" (UID: "ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:04:41.479739 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.479711    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeReady"
Jul 29 21:04:41.479927 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.479859    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 21:04:41.523429 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.523402    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7hv9z"]
Jul 29 21:04:41.525587 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.525391    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:41.526116 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.526088    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lwwqd"]
Jul 29 21:04:41.528127 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.528104    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lwwqd"
Jul 29 21:04:41.528250 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.528160    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 21:04:41.528250 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.528177    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5hxhk\""
Jul 29 21:04:41.528250 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.528210    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 21:04:41.530537 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.530515    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 21:04:41.531926 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.531862    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 21:04:41.532122 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.532103    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 21:04:41.532195 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.532139    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tnkqz\""
Jul 29 21:04:41.535730 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.535707    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7hv9z"]
Jul 29 21:04:41.538588 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.538566    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lwwqd"]
Jul 29 21:04:41.654195 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.654156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17ed28ff-e083-4d5e-819c-142ff66b85dd-tmp-dir\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:41.654389 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.654205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17ed28ff-e083-4d5e-819c-142ff66b85dd-config-volume\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:41.654389 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.654223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f60fcc10-a90b-4f77-bd05-f23f5e94fa69-cert\") pod \"ingress-canary-lwwqd\" (UID: \"f60fcc10-a90b-4f77-bd05-f23f5e94fa69\") " pod="openshift-ingress-canary/ingress-canary-lwwqd"
Jul 29 21:04:41.654389 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.654240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxcn4\" (UniqueName: \"kubernetes.io/projected/f60fcc10-a90b-4f77-bd05-f23f5e94fa69-kube-api-access-zxcn4\") pod \"ingress-canary-lwwqd\" (UID: \"f60fcc10-a90b-4f77-bd05-f23f5e94fa69\") " pod="openshift-ingress-canary/ingress-canary-lwwqd"
Jul 29 21:04:41.654389 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.654280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:41.654389 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.654372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p496\" (UniqueName: \"kubernetes.io/projected/17ed28ff-e083-4d5e-819c-142ff66b85dd-kube-api-access-7p496\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:41.709361 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.709326    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:04:41.712508 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.712486    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 21:04:41.754772 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.754741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p496\" (UniqueName: \"kubernetes.io/projected/17ed28ff-e083-4d5e-819c-142ff66b85dd-kube-api-access-7p496\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:41.754943 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.754797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17ed28ff-e083-4d5e-819c-142ff66b85dd-tmp-dir\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:41.754943 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.754872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17ed28ff-e083-4d5e-819c-142ff66b85dd-config-volume\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:41.754943 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.754910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f60fcc10-a90b-4f77-bd05-f23f5e94fa69-cert\") pod \"ingress-canary-lwwqd\" (UID: \"f60fcc10-a90b-4f77-bd05-f23f5e94fa69\") " pod="openshift-ingress-canary/ingress-canary-lwwqd"
Jul 29 21:04:41.754943 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.754939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxcn4\" (UniqueName: \"kubernetes.io/projected/f60fcc10-a90b-4f77-bd05-f23f5e94fa69-kube-api-access-zxcn4\") pod \"ingress-canary-lwwqd\" (UID: \"f60fcc10-a90b-4f77-bd05-f23f5e94fa69\") " pod="openshift-ingress-canary/ingress-canary-lwwqd"
Jul 29 21:04:41.755166 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.754968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:41.755166 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:41.755090    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:04:41.755166 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:41.755145    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls podName:17ed28ff-e083-4d5e-819c-142ff66b85dd nodeName:}" failed. No retries permitted until 2026-07-29 21:04:42.255127885 +0000 UTC m=+34.136402020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls") pod "dns-default-7hv9z" (UID: "17ed28ff-e083-4d5e-819c-142ff66b85dd") : secret "dns-default-metrics-tls" not found
Jul 29 21:04:41.755166 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.755145    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17ed28ff-e083-4d5e-819c-142ff66b85dd-tmp-dir\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:41.755540 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.755515    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17ed28ff-e083-4d5e-819c-142ff66b85dd-config-volume\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:41.759985 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.759820    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f60fcc10-a90b-4f77-bd05-f23f5e94fa69-cert\") pod \"ingress-canary-lwwqd\" (UID: \"f60fcc10-a90b-4f77-bd05-f23f5e94fa69\") " pod="openshift-ingress-canary/ingress-canary-lwwqd"
Jul 29 21:04:41.764370 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.764350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p496\" (UniqueName: \"kubernetes.io/projected/17ed28ff-e083-4d5e-819c-142ff66b85dd-kube-api-access-7p496\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:41.764517 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.764408    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxcn4\" (UniqueName: \"kubernetes.io/projected/f60fcc10-a90b-4f77-bd05-f23f5e94fa69-kube-api-access-zxcn4\") pod \"ingress-canary-lwwqd\" (UID: \"f60fcc10-a90b-4f77-bd05-f23f5e94fa69\") " pod="openshift-ingress-canary/ingress-canary-lwwqd"
Jul 29 21:04:41.848035 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.848001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lwwqd"
Jul 29 21:04:41.998063 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:41.997987    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lwwqd"]
Jul 29 21:04:42.003361 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:04:42.002423    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf60fcc10_a90b_4f77_bd05_f23f5e94fa69.slice/crio-c5fb113e709dae62887fbbf28d0b960955fa82ac683a7aadb330aa8c22ac4125 WatchSource:0}: Error finding container c5fb113e709dae62887fbbf28d0b960955fa82ac683a7aadb330aa8c22ac4125: Status 404 returned error can't find the container with id c5fb113e709dae62887fbbf28d0b960955fa82ac683a7aadb330aa8c22ac4125
Jul 29 21:04:42.260092 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:42.260001    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:42.260269 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:42.260143    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:04:42.260269 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:42.260203    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls podName:17ed28ff-e083-4d5e-819c-142ff66b85dd nodeName:}" failed. No retries permitted until 2026-07-29 21:04:43.2601852 +0000 UTC m=+35.141459324 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls") pod "dns-default-7hv9z" (UID: "17ed28ff-e083-4d5e-819c-142ff66b85dd") : secret "dns-default-metrics-tls" not found
Jul 29 21:04:42.709442 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:42.709403    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:04:42.709625 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:42.709451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:04:42.712901 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:42.712874    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 21:04:42.714085 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:42.714062    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 21:04:42.714228 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:42.714126    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cnnw5\""
Jul 29 21:04:42.714362 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:42.714068    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-82zmz\""
Jul 29 21:04:42.714362 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:42.714337    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 21:04:42.996393 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:42.996309    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lwwqd" event={"ID":"f60fcc10-a90b-4f77-bd05-f23f5e94fa69","Type":"ContainerStarted","Data":"c5fb113e709dae62887fbbf28d0b960955fa82ac683a7aadb330aa8c22ac4125"}
Jul 29 21:04:43.267672 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:43.267587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:43.267829 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:43.267764    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:04:43.267971 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:43.267852    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls podName:17ed28ff-e083-4d5e-819c-142ff66b85dd nodeName:}" failed. No retries permitted until 2026-07-29 21:04:45.267820186 +0000 UTC m=+37.149094308 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls") pod "dns-default-7hv9z" (UID: "17ed28ff-e083-4d5e-819c-142ff66b85dd") : secret "dns-default-metrics-tls" not found
Jul 29 21:04:45.281997 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:45.281967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:45.282552 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:45.282118    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:04:45.282552 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:45.282178    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls podName:17ed28ff-e083-4d5e-819c-142ff66b85dd nodeName:}" failed. No retries permitted until 2026-07-29 21:04:49.282163349 +0000 UTC m=+41.163437484 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls") pod "dns-default-7hv9z" (UID: "17ed28ff-e083-4d5e-819c-142ff66b85dd") : secret "dns-default-metrics-tls" not found
Jul 29 21:04:46.006116 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:46.005863    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lwwqd" event={"ID":"f60fcc10-a90b-4f77-bd05-f23f5e94fa69","Type":"ContainerStarted","Data":"0d0cd615500e6e7e89da5c29188c6d99b9ca61deb9746858ad69c885dd865094"}
Jul 29 21:04:46.009269 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:46.009239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jk6th" event={"ID":"7a5822af-8e5a-4b6b-9797-38c1b32049b8","Type":"ContainerStarted","Data":"ce9a6ec3f85c6812bab2eecca3cffabd0cf94e90c4b026809d32724009b0647b"}
Jul 29 21:04:46.022822 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:46.022781    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lwwqd" podStartSLOduration=1.190489036 podStartE2EDuration="5.022764927s" totalImagesPullingTime="3.832275891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:04:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:46.021695298 +0000 UTC m=+37.902969449" watchObservedRunningTime="2026-07-29 21:04:46.022764927 +0000 UTC m=+37.904039058"
Jul 29 21:04:47.014455 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:47.014426    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a5822af-8e5a-4b6b-9797-38c1b32049b8" containerID="ce9a6ec3f85c6812bab2eecca3cffabd0cf94e90c4b026809d32724009b0647b" exitCode=0
Jul 29 21:04:47.014798 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:47.014510    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jk6th" event={"ID":"7a5822af-8e5a-4b6b-9797-38c1b32049b8","Type":"ContainerDied","Data":"ce9a6ec3f85c6812bab2eecca3cffabd0cf94e90c4b026809d32724009b0647b"}
Jul 29 21:04:48.019817 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:48.019786    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a5822af-8e5a-4b6b-9797-38c1b32049b8" containerID="a658efc1190ae9461db792943df9c5fed4116ab5e3ba162d03dd0c8cf596d27d" exitCode=0
Jul 29 21:04:48.020196 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:48.019860    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jk6th" event={"ID":"7a5822af-8e5a-4b6b-9797-38c1b32049b8","Type":"ContainerDied","Data":"a658efc1190ae9461db792943df9c5fed4116ab5e3ba162d03dd0c8cf596d27d"}
Jul 29 21:04:49.025852 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:49.025802    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jk6th" event={"ID":"7a5822af-8e5a-4b6b-9797-38c1b32049b8","Type":"ContainerStarted","Data":"16f4eabc6c6df26fe2e334cb3cee6dd3478bea05ab3a01d1715725126793c350"}
Jul 29 21:04:49.048943 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:49.048892    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jk6th" podStartSLOduration=8.376121152 podStartE2EDuration="40.048876853s" totalImagesPullingTime="31.672755701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:04:49.048732192 +0000 UTC m=+40.930006349" watchObservedRunningTime="2026-07-29 21:04:49.048876853 +0000 UTC m=+40.930150990"
Jul 29 21:04:49.313165 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:49.313069    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:49.313315 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:49.313216    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:04:49.313315 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:49.313282    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls podName:17ed28ff-e083-4d5e-819c-142ff66b85dd nodeName:}" failed. No retries permitted until 2026-07-29 21:04:57.31326515 +0000 UTC m=+49.194539268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls") pod "dns-default-7hv9z" (UID: "17ed28ff-e083-4d5e-819c-142ff66b85dd") : secret "dns-default-metrics-tls" not found
Jul 29 21:04:57.371681 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:04:57.371640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:04:57.372175 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:57.371806    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:04:57.372175 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:04:57.371895    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls podName:17ed28ff-e083-4d5e-819c-142ff66b85dd nodeName:}" failed. No retries permitted until 2026-07-29 21:05:13.371875916 +0000 UTC m=+65.253150049 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls") pod "dns-default-7hv9z" (UID: "17ed28ff-e083-4d5e-819c-142ff66b85dd") : secret "dns-default-metrics-tls" not found
Jul 29 21:05:08.991992 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:08.991960    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4bvn"
Jul 29 21:05:13.384613 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.384568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:05:13.384613 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.384615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:05:13.385131 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.384645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:05:13.385131 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.384672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn6z6\" (UniqueName: \"kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6\") pod \"network-check-target-rgt2l\" (UID: \"b23d3306-5c16-4423-b2e0-28eec2615c61\") " pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:05:13.385131 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:05:13.384739    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:05:13.385131 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:05:13.384832    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls podName:17ed28ff-e083-4d5e-819c-142ff66b85dd nodeName:}" failed. No retries permitted until 2026-07-29 21:05:45.384808369 +0000 UTC m=+97.266082504 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls") pod "dns-default-7hv9z" (UID: "17ed28ff-e083-4d5e-819c-142ff66b85dd") : secret "dns-default-metrics-tls" not found
Jul 29 21:05:13.387810 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.387792    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 21:05:13.387876 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.387810    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 21:05:13.387876 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.387821    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 21:05:13.395075 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:05:13.395055    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 21:05:13.395197 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:05:13.395118    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs podName:ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b nodeName:}" failed. No retries permitted until 2026-07-29 21:06:17.3950987 +0000 UTC m=+129.276372818 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs") pod "network-metrics-daemon-6m4sz" (UID: "ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b") : secret "metrics-daemon-secret" not found
Jul 29 21:05:13.397414 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.397397    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 21:05:13.398364 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.398347    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3-original-pull-secret\") pod \"global-pull-secret-syncer-phqpp\" (UID: \"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3\") " pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:05:13.407639 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.407616    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn6z6\" (UniqueName: \"kubernetes.io/projected/b23d3306-5c16-4423-b2e0-28eec2615c61-kube-api-access-zn6z6\") pod \"network-check-target-rgt2l\" (UID: \"b23d3306-5c16-4423-b2e0-28eec2615c61\") " pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:05:13.519038 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.519001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phqpp"
Jul 29 21:05:13.634394 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.634366    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cnnw5\""
Jul 29 21:05:13.641936 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.641869    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:05:13.647509 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.647483    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-phqpp"]
Jul 29 21:05:13.651096 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:05:13.651059    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3c9b5cc_c1df_42ab_81d3_ee6dc4b1b9a3.slice/crio-4655f4b137e8a512d387bcb54468149ce710278c50605449c4c33a818b337a39 WatchSource:0}: Error finding container 4655f4b137e8a512d387bcb54468149ce710278c50605449c4c33a818b337a39: Status 404 returned error can't find the container with id 4655f4b137e8a512d387bcb54468149ce710278c50605449c4c33a818b337a39
Jul 29 21:05:13.768952 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:13.768921    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rgt2l"]
Jul 29 21:05:13.772060 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:05:13.772031    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb23d3306_5c16_4423_b2e0_28eec2615c61.slice/crio-1f35189d2ef9ba8a5557c0382affc847b83563970fc11524c625f22f5669188d WatchSource:0}: Error finding container 1f35189d2ef9ba8a5557c0382affc847b83563970fc11524c625f22f5669188d: Status 404 returned error can't find the container with id 1f35189d2ef9ba8a5557c0382affc847b83563970fc11524c625f22f5669188d
Jul 29 21:05:14.088114 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:14.088018    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rgt2l" event={"ID":"b23d3306-5c16-4423-b2e0-28eec2615c61","Type":"ContainerStarted","Data":"1f35189d2ef9ba8a5557c0382affc847b83563970fc11524c625f22f5669188d"}
Jul 29 21:05:14.089796 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:14.089763    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-phqpp" event={"ID":"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3","Type":"ContainerStarted","Data":"4655f4b137e8a512d387bcb54468149ce710278c50605449c4c33a818b337a39"}
Jul 29 21:05:17.099736 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:17.099696    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rgt2l" event={"ID":"b23d3306-5c16-4423-b2e0-28eec2615c61","Type":"ContainerStarted","Data":"0bcbc990b045977b2d832d1c752e58de82265c95f57a31742c00e692b9161721"}
Jul 29 21:05:17.100221 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:17.099826    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:05:17.116868 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:17.116801    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rgt2l" podStartSLOduration=65.018153706 podStartE2EDuration="1m8.116782803s" totalImagesPullingTime="3.098629097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:05:17.116644069 +0000 UTC m=+68.997918231" watchObservedRunningTime="2026-07-29 21:05:17.116782803 +0000 UTC m=+68.998056943"
Jul 29 21:05:21.111235 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:21.111145    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-phqpp" event={"ID":"f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3","Type":"ContainerStarted","Data":"7be78ec4ac33b9d63c7b93428c20a1455ff32f1ddb17a30dc42b5314e3237c54"}
Jul 29 21:05:21.126957 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:21.126910    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-phqpp" podStartSLOduration=65.044165213 podStartE2EDuration="1m12.126897075s" totalImagesPullingTime="7.082731862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:05:21.126114038 +0000 UTC m=+73.007388175" watchObservedRunningTime="2026-07-29 21:05:21.126897075 +0000 UTC m=+73.008171212"
Jul 29 21:05:45.396899 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:45.396744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:05:45.397295 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:05:45.396937    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:05:45.397295 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:05:45.397018    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls podName:17ed28ff-e083-4d5e-819c-142ff66b85dd nodeName:}" failed. No retries permitted until 2026-07-29 21:06:49.396998936 +0000 UTC m=+161.278273052 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls") pod "dns-default-7hv9z" (UID: "17ed28ff-e083-4d5e-819c-142ff66b85dd") : secret "dns-default-metrics-tls" not found
Jul 29 21:05:48.105245 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:05:48.105214    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rgt2l"
Jul 29 21:06:08.284615 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.284579    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j"]
Jul 29 21:06:08.286736 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.286715    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j"
Jul 29 21:06:08.289789 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.289754    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 21:06:08.289929 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.289907    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 21:06:08.291050 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.291032    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-b8gjs\""
Jul 29 21:06:08.291050 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.291041    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 21:06:08.291215 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.291186    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 29 21:06:08.300007 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.299984    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j"]
Jul 29 21:06:08.344734 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.344706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d78th\" (UniqueName: \"kubernetes.io/projected/d154453e-9253-4b19-a42c-26c3a584b670-kube-api-access-d78th\") pod \"managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j\" (UID: \"d154453e-9253-4b19-a42c-26c3a584b670\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j"
Jul 29 21:06:08.344906 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.344762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d154453e-9253-4b19-a42c-26c3a584b670-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j\" (UID: \"d154453e-9253-4b19-a42c-26c3a584b670\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j"
Jul 29 21:06:08.446097 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.446044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d78th\" (UniqueName: \"kubernetes.io/projected/d154453e-9253-4b19-a42c-26c3a584b670-kube-api-access-d78th\") pod \"managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j\" (UID: \"d154453e-9253-4b19-a42c-26c3a584b670\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j"
Jul 29 21:06:08.446097 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.446111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d154453e-9253-4b19-a42c-26c3a584b670-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j\" (UID: \"d154453e-9253-4b19-a42c-26c3a584b670\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j"
Jul 29 21:06:08.448681 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.448647    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d154453e-9253-4b19-a42c-26c3a584b670-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j\" (UID: \"d154453e-9253-4b19-a42c-26c3a584b670\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j"
Jul 29 21:06:08.454962 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.454936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d78th\" (UniqueName: \"kubernetes.io/projected/d154453e-9253-4b19-a42c-26c3a584b670-kube-api-access-d78th\") pod \"managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j\" (UID: \"d154453e-9253-4b19-a42c-26c3a584b670\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j"
Jul 29 21:06:08.605949 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.605857    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 21:06:08.616291 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.616262    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-b8gjs\""
Jul 29 21:06:08.623711 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.623675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j"
Jul 29 21:06:08.746860 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:08.746817    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j"]
Jul 29 21:06:08.750275 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:06:08.750244    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd154453e_9253_4b19_a42c_26c3a584b670.slice/crio-0980c3113be645d0ca7ac4a43b1010e438764a9ca2ba3c8890316567d9f33d1c WatchSource:0}: Error finding container 0980c3113be645d0ca7ac4a43b1010e438764a9ca2ba3c8890316567d9f33d1c: Status 404 returned error can't find the container with id 0980c3113be645d0ca7ac4a43b1010e438764a9ca2ba3c8890316567d9f33d1c
Jul 29 21:06:09.232730 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:09.232690    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j" event={"ID":"d154453e-9253-4b19-a42c-26c3a584b670","Type":"ContainerStarted","Data":"0980c3113be645d0ca7ac4a43b1010e438764a9ca2ba3c8890316567d9f33d1c"}
Jul 29 21:06:12.242853 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:12.242795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j" event={"ID":"d154453e-9253-4b19-a42c-26c3a584b670","Type":"ContainerStarted","Data":"157ddb3436eafcf1a3b382560d7286097660d9d85c53ca6acb8bc4027a89b182"}
Jul 29 21:06:12.260336 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:12.260287    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j" podStartSLOduration=1.340173837 podStartE2EDuration="4.260270883s" totalImagesPullingTime="2.920097046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:06:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:06:12.258947606 +0000 UTC m=+124.140221755" watchObservedRunningTime="2026-07-29 21:06:12.260270883 +0000 UTC m=+124.141545021"
Jul 29 21:06:17.411606 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:17.411556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:06:17.412138 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:06:17.411728    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 21:06:17.412138 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:06:17.411791    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs podName:ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b nodeName:}" failed. No retries permitted until 2026-07-29 21:08:19.411775027 +0000 UTC m=+251.293049147 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs") pod "network-metrics-daemon-6m4sz" (UID: "ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b") : secret "metrics-daemon-secret" not found
Jul 29 21:06:44.541850 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:06:44.541794    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7hv9z" podUID="17ed28ff-e083-4d5e-819c-142ff66b85dd"
Jul 29 21:06:45.324752 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:45.324725    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:06:45.723539 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:06:45.723499    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6m4sz" podUID="ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b"
Jul 29 21:06:49.445705 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:49.445668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:06:49.446211 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:06:49.445826    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:06:49.446211 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:06:49.445907    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls podName:17ed28ff-e083-4d5e-819c-142ff66b85dd nodeName:}" failed. No retries permitted until 2026-07-29 21:08:51.445889492 +0000 UTC m=+283.327163608 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls") pod "dns-default-7hv9z" (UID: "17ed28ff-e083-4d5e-819c-142ff66b85dd") : secret "dns-default-metrics-tls" not found
Jul 29 21:06:56.709607 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:06:56.709568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:07:39.383052 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.383016    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5dd67cb7f-4wd8g"]
Jul 29 21:07:39.385760 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.385743    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.388654 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.388619    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 29 21:07:39.388654 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.388636    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 29 21:07:39.388856 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.388694    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 29 21:07:39.389966 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.389946    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 29 21:07:39.389966 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.389960    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 21:07:39.390133 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.389992    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 21:07:39.390133 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.390057    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jdq4f\""
Jul 29 21:07:39.403530 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.401976    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5dd67cb7f-4wd8g"]
Jul 29 21:07:39.472852 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.472819    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"]
Jul 29 21:07:39.473119 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.473100    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz6g5\" (UniqueName: \"kubernetes.io/projected/f314863c-51a2-4c85-aa0b-96f18234f702-kube-api-access-tz6g5\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.473176 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.473145    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.473230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.473188    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.473230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.473206    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-stats-auth\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.473310 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.473233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-default-certificate\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.475604 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.475588    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"
Jul 29 21:07:39.478961 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.478937    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-ssvr7"]
Jul 29 21:07:39.480105 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.480065    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 29 21:07:39.480105 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.480096    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 29 21:07:39.480307 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.480107    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 29 21:07:39.480307 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.480097    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 29 21:07:39.480307 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.480163    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fvs7h\""
Jul 29 21:07:39.482037 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.482019    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"]
Jul 29 21:07:39.482133 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.482116    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.484562 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.484546    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 29 21:07:39.484798 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.484774    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64b8979875-77628"]
Jul 29 21:07:39.484910 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.484895    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"
Jul 29 21:07:39.487741 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.487722    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.492999 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.492974    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-s2l2g\""
Jul 29 21:07:39.493248 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.493227    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 29 21:07:39.493547 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.493527    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 21:07:39.493636 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.493564    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n9ncd\""
Jul 29 21:07:39.495794 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.495772    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"]
Jul 29 21:07:39.496907 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.496887    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-ssvr7"]
Jul 29 21:07:39.498465 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.498447    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 21:07:39.500292 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.500270    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 29 21:07:39.500613 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.500557    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 29 21:07:39.500854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.500823    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 21:07:39.501437 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.501418    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 29 21:07:39.501516 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.501478    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"]
Jul 29 21:07:39.501570 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.501556    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-4xfwc\""
Jul 29 21:07:39.505561 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.505542    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 29 21:07:39.506565 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.506550    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 29 21:07:39.506629 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.506569    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 29 21:07:39.506787 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.506772    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 29 21:07:39.516195 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.516177    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 21:07:39.519022 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.518995    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64b8979875-77628"]
Jul 29 21:07:39.573989 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.573957    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.573989 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.573992    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42ac4b00-61be-4468-8b6d-e85603e33b67-config\") pod \"console-operator-584dcb6b54-ssvr7\" (UID: \"42ac4b00-61be-4468-8b6d-e85603e33b67\") " pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.574199 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42ac4b00-61be-4468-8b6d-e85603e33b67-serving-cert\") pod \"console-operator-584dcb6b54-ssvr7\" (UID: \"42ac4b00-61be-4468-8b6d-e85603e33b67\") " pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.574199 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.574199 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-stats-auth\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.574298 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-image-registry-private-configuration\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.574298 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:39.574211    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle podName:f314863c-51a2-4c85-aa0b-96f18234f702 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:40.074190563 +0000 UTC m=+211.955464696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle") pod "router-default-5dd67cb7f-4wd8g" (UID: "f314863c-51a2-4c85-aa0b-96f18234f702") : configmap references non-existent config key: service-ca.crt
Jul 29 21:07:39.574298 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574247    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-default-certificate\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.574298 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-certificates\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.574458 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574300    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzx85\" (UniqueName: \"kubernetes.io/projected/ffee65c3-a94f-436b-907e-f5c81f28752f-kube-api-access-fzx85\") pod \"service-ca-operator-5d6c75f84f-h7x8h\" (UID: \"ffee65c3-a94f-436b-907e-f5c81f28752f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"
Jul 29 21:07:39.574458 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574421    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ffee65c3-a94f-436b-907e-f5c81f28752f-config\") pod \"service-ca-operator-5d6c75f84f-h7x8h\" (UID: \"ffee65c3-a94f-436b-907e-f5c81f28752f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"
Jul 29 21:07:39.574458 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-installation-pull-secrets\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.574564 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574491    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-bound-sa-token\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.574564 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574526    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4hh9\" (UniqueName: \"kubernetes.io/projected/d135c4fb-0ebd-485a-8dbb-39369e431db7-kube-api-access-r4hh9\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg\" (UID: \"d135c4fb-0ebd-485a-8dbb-39369e431db7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"
Jul 29 21:07:39.574564 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574554    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ffee65c3-a94f-436b-907e-f5c81f28752f-serving-cert\") pod \"service-ca-operator-5d6c75f84f-h7x8h\" (UID: \"ffee65c3-a94f-436b-907e-f5c81f28752f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"
Jul 29 21:07:39.574698 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d135c4fb-0ebd-485a-8dbb-39369e431db7-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg\" (UID: \"d135c4fb-0ebd-485a-8dbb-39369e431db7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"
Jul 29 21:07:39.574698 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574592    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/42ac4b00-61be-4468-8b6d-e85603e33b67-trusted-ca\") pod \"console-operator-584dcb6b54-ssvr7\" (UID: \"42ac4b00-61be-4468-8b6d-e85603e33b67\") " pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.574794 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz6g5\" (UniqueName: \"kubernetes.io/projected/f314863c-51a2-4c85-aa0b-96f18234f702-kube-api-access-tz6g5\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.574794 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574758    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d135c4fb-0ebd-485a-8dbb-39369e431db7-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg\" (UID: \"d135c4fb-0ebd-485a-8dbb-39369e431db7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"
Jul 29 21:07:39.574982 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.574815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pph2b\" (UniqueName: \"kubernetes.io/projected/42ac4b00-61be-4468-8b6d-e85603e33b67-kube-api-access-pph2b\") pod \"console-operator-584dcb6b54-ssvr7\" (UID: \"42ac4b00-61be-4468-8b6d-e85603e33b67\") " pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.575099 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.575016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-ca-trust-extracted\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.575099 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.575065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-trusted-ca\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.575099 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.575094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhpjw\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-kube-api-access-qhpjw\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.575214 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.575164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.575311 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:39.575294    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 21:07:39.575384 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:39.575373    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs podName:f314863c-51a2-4c85-aa0b-96f18234f702 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:40.075354782 +0000 UTC m=+211.956628903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs") pod "router-default-5dd67cb7f-4wd8g" (UID: "f314863c-51a2-4c85-aa0b-96f18234f702") : secret "router-metrics-certs-default" not found
Jul 29 21:07:39.576597 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.576578    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-stats-auth\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.576730 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.576709    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-default-certificate\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.584275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.584254    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz6g5\" (UniqueName: \"kubernetes.io/projected/f314863c-51a2-4c85-aa0b-96f18234f702-kube-api-access-tz6g5\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:39.675918 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.675887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.676078 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.675924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42ac4b00-61be-4468-8b6d-e85603e33b67-config\") pod \"console-operator-584dcb6b54-ssvr7\" (UID: \"42ac4b00-61be-4468-8b6d-e85603e33b67\") " pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.676078 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.675939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42ac4b00-61be-4468-8b6d-e85603e33b67-serving-cert\") pod \"console-operator-584dcb6b54-ssvr7\" (UID: \"42ac4b00-61be-4468-8b6d-e85603e33b67\") " pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.676149 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:39.676096    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 21:07:39.676149 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-image-registry-private-configuration\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.676149 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-certificates\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.676264 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:39.676113    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b8979875-77628: secret "image-registry-tls" not found
Jul 29 21:07:39.676264 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzx85\" (UniqueName: \"kubernetes.io/projected/ffee65c3-a94f-436b-907e-f5c81f28752f-kube-api-access-fzx85\") pod \"service-ca-operator-5d6c75f84f-h7x8h\" (UID: \"ffee65c3-a94f-436b-907e-f5c81f28752f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"
Jul 29 21:07:39.676264 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ffee65c3-a94f-436b-907e-f5c81f28752f-config\") pod \"service-ca-operator-5d6c75f84f-h7x8h\" (UID: \"ffee65c3-a94f-436b-907e-f5c81f28752f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"
Jul 29 21:07:39.676264 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:39.676225    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls podName:cdb47f20-85a3-485c-9b7d-acd9000f2a1a nodeName:}" failed. No retries permitted until 2026-07-29 21:07:40.176202405 +0000 UTC m=+212.057476544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls") pod "image-registry-64b8979875-77628" (UID: "cdb47f20-85a3-485c-9b7d-acd9000f2a1a") : secret "image-registry-tls" not found
Jul 29 21:07:39.676460 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-installation-pull-secrets\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.676460 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-bound-sa-token\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.676460 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676335    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4hh9\" (UniqueName: \"kubernetes.io/projected/d135c4fb-0ebd-485a-8dbb-39369e431db7-kube-api-access-r4hh9\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg\" (UID: \"d135c4fb-0ebd-485a-8dbb-39369e431db7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"
Jul 29 21:07:39.676460 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ffee65c3-a94f-436b-907e-f5c81f28752f-serving-cert\") pod \"service-ca-operator-5d6c75f84f-h7x8h\" (UID: \"ffee65c3-a94f-436b-907e-f5c81f28752f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"
Jul 29 21:07:39.676460 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d135c4fb-0ebd-485a-8dbb-39369e431db7-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg\" (UID: \"d135c4fb-0ebd-485a-8dbb-39369e431db7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"
Jul 29 21:07:39.676460 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/42ac4b00-61be-4468-8b6d-e85603e33b67-trusted-ca\") pod \"console-operator-584dcb6b54-ssvr7\" (UID: \"42ac4b00-61be-4468-8b6d-e85603e33b67\") " pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.676740 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676489    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d135c4fb-0ebd-485a-8dbb-39369e431db7-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg\" (UID: \"d135c4fb-0ebd-485a-8dbb-39369e431db7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"
Jul 29 21:07:39.676740 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pph2b\" (UniqueName: \"kubernetes.io/projected/42ac4b00-61be-4468-8b6d-e85603e33b67-kube-api-access-pph2b\") pod \"console-operator-584dcb6b54-ssvr7\" (UID: \"42ac4b00-61be-4468-8b6d-e85603e33b67\") " pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.676740 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-ca-trust-extracted\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.676740 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-trusted-ca\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.676740 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhpjw\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-kube-api-access-qhpjw\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.676740 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ffee65c3-a94f-436b-907e-f5c81f28752f-config\") pod \"service-ca-operator-5d6c75f84f-h7x8h\" (UID: \"ffee65c3-a94f-436b-907e-f5c81f28752f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"
Jul 29 21:07:39.676740 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676730    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42ac4b00-61be-4468-8b6d-e85603e33b67-config\") pod \"console-operator-584dcb6b54-ssvr7\" (UID: \"42ac4b00-61be-4468-8b6d-e85603e33b67\") " pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.677131 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.676811    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-certificates\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.677276 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.677148    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-ca-trust-extracted\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.677720 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.677696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d135c4fb-0ebd-485a-8dbb-39369e431db7-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg\" (UID: \"d135c4fb-0ebd-485a-8dbb-39369e431db7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"
Jul 29 21:07:39.677974 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.677947    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-trusted-ca\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.678101 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.677953    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/42ac4b00-61be-4468-8b6d-e85603e33b67-trusted-ca\") pod \"console-operator-584dcb6b54-ssvr7\" (UID: \"42ac4b00-61be-4468-8b6d-e85603e33b67\") " pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.678784 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.678761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42ac4b00-61be-4468-8b6d-e85603e33b67-serving-cert\") pod \"console-operator-584dcb6b54-ssvr7\" (UID: \"42ac4b00-61be-4468-8b6d-e85603e33b67\") " pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.679198 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.679174    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d135c4fb-0ebd-485a-8dbb-39369e431db7-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg\" (UID: \"d135c4fb-0ebd-485a-8dbb-39369e431db7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"
Jul 29 21:07:39.679292 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.679226    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-installation-pull-secrets\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.679498 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.679480    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ffee65c3-a94f-436b-907e-f5c81f28752f-serving-cert\") pod \"service-ca-operator-5d6c75f84f-h7x8h\" (UID: \"ffee65c3-a94f-436b-907e-f5c81f28752f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"
Jul 29 21:07:39.679818 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.679798    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-image-registry-private-configuration\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.688510 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.688486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4hh9\" (UniqueName: \"kubernetes.io/projected/d135c4fb-0ebd-485a-8dbb-39369e431db7-kube-api-access-r4hh9\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg\" (UID: \"d135c4fb-0ebd-485a-8dbb-39369e431db7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"
Jul 29 21:07:39.688628 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.688605    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-bound-sa-token\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.688680 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.688665    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzx85\" (UniqueName: \"kubernetes.io/projected/ffee65c3-a94f-436b-907e-f5c81f28752f-kube-api-access-fzx85\") pod \"service-ca-operator-5d6c75f84f-h7x8h\" (UID: \"ffee65c3-a94f-436b-907e-f5c81f28752f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"
Jul 29 21:07:39.689317 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.689296    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhpjw\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-kube-api-access-qhpjw\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:39.689764 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.689747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pph2b\" (UniqueName: \"kubernetes.io/projected/42ac4b00-61be-4468-8b6d-e85603e33b67-kube-api-access-pph2b\") pod \"console-operator-584dcb6b54-ssvr7\" (UID: \"42ac4b00-61be-4468-8b6d-e85603e33b67\") " pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.785445 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.785414    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"
Jul 29 21:07:39.792787 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.792767    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:39.800508 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.800482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"
Jul 29 21:07:39.930701 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:39.930526    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg"]
Jul 29 21:07:39.933182 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:07:39.933157    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd135c4fb_0ebd_485a_8dbb_39369e431db7.slice/crio-0b4fe80649a9b2ee1b780326dab93fdeab601d009340cbc78a9981355ab6875a WatchSource:0}: Error finding container 0b4fe80649a9b2ee1b780326dab93fdeab601d009340cbc78a9981355ab6875a: Status 404 returned error can't find the container with id 0b4fe80649a9b2ee1b780326dab93fdeab601d009340cbc78a9981355ab6875a
Jul 29 21:07:40.079514 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:40.079472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:40.079694 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:40.079525    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:40.079694 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:40.079634    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 21:07:40.079694 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:40.079659    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle podName:f314863c-51a2-4c85-aa0b-96f18234f702 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:41.079643574 +0000 UTC m=+212.960917690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle") pod "router-default-5dd67cb7f-4wd8g" (UID: "f314863c-51a2-4c85-aa0b-96f18234f702") : configmap references non-existent config key: service-ca.crt
Jul 29 21:07:40.079694 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:40.079694    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs podName:f314863c-51a2-4c85-aa0b-96f18234f702 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:41.079678652 +0000 UTC m=+212.960952767 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs") pod "router-default-5dd67cb7f-4wd8g" (UID: "f314863c-51a2-4c85-aa0b-96f18234f702") : secret "router-metrics-certs-default" not found
Jul 29 21:07:40.151196 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:40.151145    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h"]
Jul 29 21:07:40.153519 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:07:40.153489    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffee65c3_a94f_436b_907e_f5c81f28752f.slice/crio-7f515ed1b1a3641083192120b2883e0e9acc4ef014a5a53a3cd3be27b1ab1d70 WatchSource:0}: Error finding container 7f515ed1b1a3641083192120b2883e0e9acc4ef014a5a53a3cd3be27b1ab1d70: Status 404 returned error can't find the container with id 7f515ed1b1a3641083192120b2883e0e9acc4ef014a5a53a3cd3be27b1ab1d70
Jul 29 21:07:40.154035 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:40.154016    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-ssvr7"]
Jul 29 21:07:40.156357 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:07:40.156329    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42ac4b00_61be_4468_8b6d_e85603e33b67.slice/crio-80a06eb5c1371d2609cccf199697af95d1fdb811b7f720d83232ad141c1d64be WatchSource:0}: Error finding container 80a06eb5c1371d2609cccf199697af95d1fdb811b7f720d83232ad141c1d64be: Status 404 returned error can't find the container with id 80a06eb5c1371d2609cccf199697af95d1fdb811b7f720d83232ad141c1d64be
Jul 29 21:07:40.180155 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:40.180130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:40.180261 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:40.180250    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 21:07:40.180311 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:40.180263    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b8979875-77628: secret "image-registry-tls" not found
Jul 29 21:07:40.180311 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:40.180304    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls podName:cdb47f20-85a3-485c-9b7d-acd9000f2a1a nodeName:}" failed. No retries permitted until 2026-07-29 21:07:41.180290181 +0000 UTC m=+213.061564296 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls") pod "image-registry-64b8979875-77628" (UID: "cdb47f20-85a3-485c-9b7d-acd9000f2a1a") : secret "image-registry-tls" not found
Jul 29 21:07:40.456102 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:40.456062    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h" event={"ID":"ffee65c3-a94f-436b-907e-f5c81f28752f","Type":"ContainerStarted","Data":"7f515ed1b1a3641083192120b2883e0e9acc4ef014a5a53a3cd3be27b1ab1d70"}
Jul 29 21:07:40.457141 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:40.457114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7" event={"ID":"42ac4b00-61be-4468-8b6d-e85603e33b67","Type":"ContainerStarted","Data":"80a06eb5c1371d2609cccf199697af95d1fdb811b7f720d83232ad141c1d64be"}
Jul 29 21:07:40.458025 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:40.458005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg" event={"ID":"d135c4fb-0ebd-485a-8dbb-39369e431db7","Type":"ContainerStarted","Data":"0b4fe80649a9b2ee1b780326dab93fdeab601d009340cbc78a9981355ab6875a"}
Jul 29 21:07:41.086948 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:41.086908    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:41.087135 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:41.086984    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:41.087135 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:41.087121    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 21:07:41.087222 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:41.087155    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle podName:f314863c-51a2-4c85-aa0b-96f18234f702 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:43.087141577 +0000 UTC m=+214.968415692 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle") pod "router-default-5dd67cb7f-4wd8g" (UID: "f314863c-51a2-4c85-aa0b-96f18234f702") : configmap references non-existent config key: service-ca.crt
Jul 29 21:07:41.087222 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:41.087202    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs podName:f314863c-51a2-4c85-aa0b-96f18234f702 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:43.08718274 +0000 UTC m=+214.968456866 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs") pod "router-default-5dd67cb7f-4wd8g" (UID: "f314863c-51a2-4c85-aa0b-96f18234f702") : secret "router-metrics-certs-default" not found
Jul 29 21:07:41.187799 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:41.187758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:41.187999 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:41.187931    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 21:07:41.187999 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:41.187954    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b8979875-77628: secret "image-registry-tls" not found
Jul 29 21:07:41.188097 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:41.188025    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls podName:cdb47f20-85a3-485c-9b7d-acd9000f2a1a nodeName:}" failed. No retries permitted until 2026-07-29 21:07:43.188003624 +0000 UTC m=+215.069277742 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls") pod "image-registry-64b8979875-77628" (UID: "cdb47f20-85a3-485c-9b7d-acd9000f2a1a") : secret "image-registry-tls" not found
Jul 29 21:07:43.105813 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.105772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:43.106299 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.105862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:43.106299 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:43.105969    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 21:07:43.106299 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:43.106032    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle podName:f314863c-51a2-4c85-aa0b-96f18234f702 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:47.106011494 +0000 UTC m=+218.987285637 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle") pod "router-default-5dd67cb7f-4wd8g" (UID: "f314863c-51a2-4c85-aa0b-96f18234f702") : configmap references non-existent config key: service-ca.crt
Jul 29 21:07:43.106299 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:43.106056    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs podName:f314863c-51a2-4c85-aa0b-96f18234f702 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:47.106045907 +0000 UTC m=+218.987320023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs") pod "router-default-5dd67cb7f-4wd8g" (UID: "f314863c-51a2-4c85-aa0b-96f18234f702") : secret "router-metrics-certs-default" not found
Jul 29 21:07:43.207042 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.207020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:43.207144 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:43.207131    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 21:07:43.207189 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:43.207146    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b8979875-77628: secret "image-registry-tls" not found
Jul 29 21:07:43.207224 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:43.207215    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls podName:cdb47f20-85a3-485c-9b7d-acd9000f2a1a nodeName:}" failed. No retries permitted until 2026-07-29 21:07:47.207174719 +0000 UTC m=+219.088448835 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls") pod "image-registry-64b8979875-77628" (UID: "cdb47f20-85a3-485c-9b7d-acd9000f2a1a") : secret "image-registry-tls" not found
Jul 29 21:07:43.312721 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.312477    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-j2scg"]
Jul 29 21:07:43.316395 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.316368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:43.321970 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.321944    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 21:07:43.322106 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.321982    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 21:07:43.322266 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.322246    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qz6qg\""
Jul 29 21:07:43.322402 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.322251    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 21:07:43.323268 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.323238    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 21:07:43.334988 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.334964    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j2scg"]
Jul 29 21:07:43.408466 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.408430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/88e24522-aa6c-43bb-8805-b8e4d682e5a2-crio-socket\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:43.408608 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.408477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:43.408666 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.408614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:43.408666 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.408644    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qf6z\" (UniqueName: \"kubernetes.io/projected/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-api-access-7qf6z\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:43.408742 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.408704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/88e24522-aa6c-43bb-8805-b8e4d682e5a2-data-volume\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:43.468819 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.468783    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg" event={"ID":"d135c4fb-0ebd-485a-8dbb-39369e431db7","Type":"ContainerStarted","Data":"e3063de80d0ecae28ba7c3bc2bcec164db61c130a2f71e626fb072427f098f52"}
Jul 29 21:07:43.470346 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.470317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h" event={"ID":"ffee65c3-a94f-436b-907e-f5c81f28752f","Type":"ContainerStarted","Data":"b04621ad5718e2a13874795e22e8fa50542bf00b5bbe832c0fa08ac5968c8495"}
Jul 29 21:07:43.471981 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.471958    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/0.log"
Jul 29 21:07:43.472084 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.471997    2580 generic.go:356] "Generic (PLEG): container finished" podID="42ac4b00-61be-4468-8b6d-e85603e33b67" containerID="d426d02201b2d053f5f39191decf685958d23657337d03435f2cbcb4548db320" exitCode=255
Jul 29 21:07:43.472084 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.472026    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7" event={"ID":"42ac4b00-61be-4468-8b6d-e85603e33b67","Type":"ContainerDied","Data":"d426d02201b2d053f5f39191decf685958d23657337d03435f2cbcb4548db320"}
Jul 29 21:07:43.472310 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.472294    2580 scope.go:117] "RemoveContainer" containerID="d426d02201b2d053f5f39191decf685958d23657337d03435f2cbcb4548db320"
Jul 29 21:07:43.485948 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.485897    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg" podStartSLOduration=1.229013978 podStartE2EDuration="4.485879642s" totalImagesPullingTime="3.256865664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:07:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:07:43.484536537 +0000 UTC m=+215.365810676" watchObservedRunningTime="2026-07-29 21:07:43.485879642 +0000 UTC m=+215.367153780"
Jul 29 21:07:43.500635 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.500592    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h" podStartSLOduration=1.456554477 podStartE2EDuration="4.500578216s" totalImagesPullingTime="3.044023739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:07:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:07:43.500337015 +0000 UTC m=+215.381611153" watchObservedRunningTime="2026-07-29 21:07:43.500578216 +0000 UTC m=+215.381852332"
Jul 29 21:07:43.509698 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.509092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:43.509698 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.509139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qf6z\" (UniqueName: \"kubernetes.io/projected/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-api-access-7qf6z\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:43.509698 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.509207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/88e24522-aa6c-43bb-8805-b8e4d682e5a2-data-volume\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:43.509698 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:43.509224    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 21:07:43.509698 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.509237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/88e24522-aa6c-43bb-8805-b8e4d682e5a2-crio-socket\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:43.509698 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.509262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:43.509698 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:43.509281    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm podName:88e24522-aa6c-43bb-8805-b8e4d682e5a2 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:44.009263652 +0000 UTC m=+215.890537786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j2scg" (UID: "88e24522-aa6c-43bb-8805-b8e4d682e5a2") : configmap "kube-rbac-proxy" not found
Jul 29 21:07:43.509698 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:43.509342    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 21:07:43.509698 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:43.509378    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls podName:88e24522-aa6c-43bb-8805-b8e4d682e5a2 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:44.009367701 +0000 UTC m=+215.890641817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j2scg" (UID: "88e24522-aa6c-43bb-8805-b8e4d682e5a2") : secret "insights-runtime-extractor-tls" not found
Jul 29 21:07:43.509698 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.509623    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/88e24522-aa6c-43bb-8805-b8e4d682e5a2-crio-socket\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:43.510254 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.509777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/88e24522-aa6c-43bb-8805-b8e4d682e5a2-data-volume\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:43.521353 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:43.521329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qf6z\" (UniqueName: \"kubernetes.io/projected/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-api-access-7qf6z\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:44.013948 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:44.013902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:44.014164 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:44.013991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:44.014164 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:44.014031    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 21:07:44.014164 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:44.014107    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm podName:88e24522-aa6c-43bb-8805-b8e4d682e5a2 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:45.014087797 +0000 UTC m=+216.895361919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j2scg" (UID: "88e24522-aa6c-43bb-8805-b8e4d682e5a2") : configmap "kube-rbac-proxy" not found
Jul 29 21:07:44.014164 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:44.014125    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 21:07:44.014164 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:44.014161    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls podName:88e24522-aa6c-43bb-8805-b8e4d682e5a2 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:45.014149711 +0000 UTC m=+216.895423828 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j2scg" (UID: "88e24522-aa6c-43bb-8805-b8e4d682e5a2") : secret "insights-runtime-extractor-tls" not found
Jul 29 21:07:44.477098 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:44.477068    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/1.log"
Jul 29 21:07:44.477523 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:44.477473    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/0.log"
Jul 29 21:07:44.477523 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:44.477508    2580 generic.go:356] "Generic (PLEG): container finished" podID="42ac4b00-61be-4468-8b6d-e85603e33b67" containerID="f009ba276a7e35bca5ce0a0ababb4982d6c683cfec2a5b893d0a0ec511203ff8" exitCode=255
Jul 29 21:07:44.477596 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:44.477581    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7" event={"ID":"42ac4b00-61be-4468-8b6d-e85603e33b67","Type":"ContainerDied","Data":"f009ba276a7e35bca5ce0a0ababb4982d6c683cfec2a5b893d0a0ec511203ff8"}
Jul 29 21:07:44.477673 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:44.477660    2580 scope.go:117] "RemoveContainer" containerID="d426d02201b2d053f5f39191decf685958d23657337d03435f2cbcb4548db320"
Jul 29 21:07:44.477993 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:44.477966    2580 scope.go:117] "RemoveContainer" containerID="f009ba276a7e35bca5ce0a0ababb4982d6c683cfec2a5b893d0a0ec511203ff8"
Jul 29 21:07:44.478180 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:44.478160    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-ssvr7_openshift-console-operator(42ac4b00-61be-4468-8b6d-e85603e33b67)\"" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7" podUID="42ac4b00-61be-4468-8b6d-e85603e33b67"
Jul 29 21:07:45.021769 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:45.021716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:45.021941 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:45.021832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:45.021941 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:45.021923    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 21:07:45.022044 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:45.021943    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 21:07:45.022044 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:45.021991    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls podName:88e24522-aa6c-43bb-8805-b8e4d682e5a2 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:47.021972694 +0000 UTC m=+218.903246812 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j2scg" (UID: "88e24522-aa6c-43bb-8805-b8e4d682e5a2") : secret "insights-runtime-extractor-tls" not found
Jul 29 21:07:45.022044 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:45.022007    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm podName:88e24522-aa6c-43bb-8805-b8e4d682e5a2 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:47.021999715 +0000 UTC m=+218.903273831 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j2scg" (UID: "88e24522-aa6c-43bb-8805-b8e4d682e5a2") : configmap "kube-rbac-proxy" not found
Jul 29 21:07:45.481604 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:45.481572    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/1.log"
Jul 29 21:07:45.482036 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:45.481938    2580 scope.go:117] "RemoveContainer" containerID="f009ba276a7e35bca5ce0a0ababb4982d6c683cfec2a5b893d0a0ec511203ff8"
Jul 29 21:07:45.482111 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:45.482084    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-ssvr7_openshift-console-operator(42ac4b00-61be-4468-8b6d-e85603e33b67)\"" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7" podUID="42ac4b00-61be-4468-8b6d-e85603e33b67"
Jul 29 21:07:45.503565 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:45.503541    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f58qv_97bd114e-b39c-4454-be0a-a964b2db56f8/dns-node-resolver/0.log"
Jul 29 21:07:46.104603 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:46.104577    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-25rxb_d44ed47a-d0fe-4536-a552-4ee051c80914/node-ca/0.log"
Jul 29 21:07:47.038105 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:47.038011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:47.038105 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:47.038077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:47.038496 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:47.038145    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 21:07:47.038496 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:47.038169    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 21:07:47.038496 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:47.038225    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm podName:88e24522-aa6c-43bb-8805-b8e4d682e5a2 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:51.038205147 +0000 UTC m=+222.919479266 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j2scg" (UID: "88e24522-aa6c-43bb-8805-b8e4d682e5a2") : configmap "kube-rbac-proxy" not found
Jul 29 21:07:47.038496 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:47.038240    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls podName:88e24522-aa6c-43bb-8805-b8e4d682e5a2 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:51.038234222 +0000 UTC m=+222.919508338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j2scg" (UID: "88e24522-aa6c-43bb-8805-b8e4d682e5a2") : secret "insights-runtime-extractor-tls" not found
Jul 29 21:07:47.139216 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:47.139176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:47.139388 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:47.139306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:47.139388 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:47.139354    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle podName:f314863c-51a2-4c85-aa0b-96f18234f702 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:55.139338149 +0000 UTC m=+227.020612269 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle") pod "router-default-5dd67cb7f-4wd8g" (UID: "f314863c-51a2-4c85-aa0b-96f18234f702") : configmap references non-existent config key: service-ca.crt
Jul 29 21:07:47.139539 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:47.139418    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 21:07:47.139539 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:47.139483    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs podName:f314863c-51a2-4c85-aa0b-96f18234f702 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:55.139458885 +0000 UTC m=+227.020733002 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs") pod "router-default-5dd67cb7f-4wd8g" (UID: "f314863c-51a2-4c85-aa0b-96f18234f702") : secret "router-metrics-certs-default" not found
Jul 29 21:07:47.240021 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:47.239985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:47.240168 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:47.240136    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 21:07:47.240168 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:47.240158    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64b8979875-77628: secret "image-registry-tls" not found
Jul 29 21:07:47.240229 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:47.240216    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls podName:cdb47f20-85a3-485c-9b7d-acd9000f2a1a nodeName:}" failed. No retries permitted until 2026-07-29 21:07:55.240200485 +0000 UTC m=+227.121474605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls") pod "image-registry-64b8979875-77628" (UID: "cdb47f20-85a3-485c-9b7d-acd9000f2a1a") : secret "image-registry-tls" not found
Jul 29 21:07:47.306834 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:47.306762    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lwwqd_f60fcc10-a90b-4f77-bd05-f23f5e94fa69/serve-healthcheck-canary/0.log"
Jul 29 21:07:49.792888 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:49.792833    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:49.793322 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:49.793010    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:07:49.793322 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:49.793266    2580 scope.go:117] "RemoveContainer" containerID="f009ba276a7e35bca5ce0a0ababb4982d6c683cfec2a5b893d0a0ec511203ff8"
Jul 29 21:07:49.793431 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:49.793411    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-ssvr7_openshift-console-operator(42ac4b00-61be-4468-8b6d-e85603e33b67)\"" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7" podUID="42ac4b00-61be-4468-8b6d-e85603e33b67"
Jul 29 21:07:50.495591 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:50.495562    2580 scope.go:117] "RemoveContainer" containerID="f009ba276a7e35bca5ce0a0ababb4982d6c683cfec2a5b893d0a0ec511203ff8"
Jul 29 21:07:50.495743 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:50.495721    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-ssvr7_openshift-console-operator(42ac4b00-61be-4468-8b6d-e85603e33b67)\"" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7" podUID="42ac4b00-61be-4468-8b6d-e85603e33b67"
Jul 29 21:07:51.071871 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:51.071821    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:51.072236 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:51.071901    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:51.072236 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:51.071928    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 21:07:51.072236 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:51.071991    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 21:07:51.072236 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:51.072002    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm podName:88e24522-aa6c-43bb-8805-b8e4d682e5a2 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:59.071983445 +0000 UTC m=+230.953257563 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j2scg" (UID: "88e24522-aa6c-43bb-8805-b8e4d682e5a2") : configmap "kube-rbac-proxy" not found
Jul 29 21:07:51.072236 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:51.072033    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls podName:88e24522-aa6c-43bb-8805-b8e4d682e5a2 nodeName:}" failed. No retries permitted until 2026-07-29 21:07:59.072021377 +0000 UTC m=+230.953295494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j2scg" (UID: "88e24522-aa6c-43bb-8805-b8e4d682e5a2") : secret "insights-runtime-extractor-tls" not found
Jul 29 21:07:55.204783 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:55.204744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:55.205258 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:55.204796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:07:55.205258 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:55.204938    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 21:07:55.205258 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:55.204953    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle podName:f314863c-51a2-4c85-aa0b-96f18234f702 nodeName:}" failed. No retries permitted until 2026-07-29 21:08:11.204936191 +0000 UTC m=+243.086210310 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle") pod "router-default-5dd67cb7f-4wd8g" (UID: "f314863c-51a2-4c85-aa0b-96f18234f702") : configmap references non-existent config key: service-ca.crt
Jul 29 21:07:55.205258 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:55.205019    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs podName:f314863c-51a2-4c85-aa0b-96f18234f702 nodeName:}" failed. No retries permitted until 2026-07-29 21:08:11.205001523 +0000 UTC m=+243.086275651 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs") pod "router-default-5dd67cb7f-4wd8g" (UID: "f314863c-51a2-4c85-aa0b-96f18234f702") : secret "router-metrics-certs-default" not found
Jul 29 21:07:55.305385 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:55.305344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:55.307874 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:55.307827    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls\") pod \"image-registry-64b8979875-77628\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") " pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:55.406518 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:55.406481    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:55.538018 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:55.537989    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64b8979875-77628"]
Jul 29 21:07:55.540239 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:07:55.540200    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdb47f20_85a3_485c_9b7d_acd9000f2a1a.slice/crio-6bf8120de1b8c7d9f5d5292ece3875a06d8877da587f814bcab646274d76ed9d WatchSource:0}: Error finding container 6bf8120de1b8c7d9f5d5292ece3875a06d8877da587f814bcab646274d76ed9d: Status 404 returned error can't find the container with id 6bf8120de1b8c7d9f5d5292ece3875a06d8877da587f814bcab646274d76ed9d
Jul 29 21:07:56.514572 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:56.514529    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b8979875-77628" event={"ID":"cdb47f20-85a3-485c-9b7d-acd9000f2a1a","Type":"ContainerStarted","Data":"4909f042124defc1d18b7bc9f3eef5d067aee7a8266e0d56d5098d9024841aba"}
Jul 29 21:07:56.514991 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:56.514626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b8979875-77628" event={"ID":"cdb47f20-85a3-485c-9b7d-acd9000f2a1a","Type":"ContainerStarted","Data":"6bf8120de1b8c7d9f5d5292ece3875a06d8877da587f814bcab646274d76ed9d"}
Jul 29 21:07:56.514991 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:56.514744    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:07:56.536533 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:56.536486    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64b8979875-77628" podStartSLOduration=17.536473794 podStartE2EDuration="17.536473794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:07:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:07:56.535795429 +0000 UTC m=+228.417069581" watchObservedRunningTime="2026-07-29 21:07:56.536473794 +0000 UTC m=+228.417747932"
Jul 29 21:07:59.131586 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:59.131545    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:59.132012 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:59.131610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:07:59.132012 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:59.131696    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 21:07:59.132012 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:07:59.131750    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls podName:88e24522-aa6c-43bb-8805-b8e4d682e5a2 nodeName:}" failed. No retries permitted until 2026-07-29 21:08:15.131735875 +0000 UTC m=+247.013009990 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j2scg" (UID: "88e24522-aa6c-43bb-8805-b8e4d682e5a2") : secret "insights-runtime-extractor-tls" not found
Jul 29 21:07:59.132141 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:07:59.132116    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/88e24522-aa6c-43bb-8805-b8e4d682e5a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:08:02.709298 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:02.709263    2580 scope.go:117] "RemoveContainer" containerID="f009ba276a7e35bca5ce0a0ababb4982d6c683cfec2a5b893d0a0ec511203ff8"
Jul 29 21:08:03.537848 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:03.537821    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:08:03.538251 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:03.538235    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/1.log"
Jul 29 21:08:03.538324 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:03.538269    2580 generic.go:356] "Generic (PLEG): container finished" podID="42ac4b00-61be-4468-8b6d-e85603e33b67" containerID="409294e1d334fb5cc079a91c4a7138d88e28b9c5f6480a2e0cb562d12907b5d7" exitCode=255
Jul 29 21:08:03.538360 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:03.538318    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7" event={"ID":"42ac4b00-61be-4468-8b6d-e85603e33b67","Type":"ContainerDied","Data":"409294e1d334fb5cc079a91c4a7138d88e28b9c5f6480a2e0cb562d12907b5d7"}
Jul 29 21:08:03.538400 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:03.538367    2580 scope.go:117] "RemoveContainer" containerID="f009ba276a7e35bca5ce0a0ababb4982d6c683cfec2a5b893d0a0ec511203ff8"
Jul 29 21:08:03.538697 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:03.538678    2580 scope.go:117] "RemoveContainer" containerID="409294e1d334fb5cc079a91c4a7138d88e28b9c5f6480a2e0cb562d12907b5d7"
Jul 29 21:08:03.538892 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:08:03.538870    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-ssvr7_openshift-console-operator(42ac4b00-61be-4468-8b6d-e85603e33b67)\"" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7" podUID="42ac4b00-61be-4468-8b6d-e85603e33b67"
Jul 29 21:08:04.542562 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:04.542537    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:08:05.992502 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:05.992473    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64b8979875-77628"]
Jul 29 21:08:09.793041 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:09.792990    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:08:09.793474 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:09.793064    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:08:09.793474 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:09.793366    2580 scope.go:117] "RemoveContainer" containerID="409294e1d334fb5cc079a91c4a7138d88e28b9c5f6480a2e0cb562d12907b5d7"
Jul 29 21:08:09.793549 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:08:09.793522    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-ssvr7_openshift-console-operator(42ac4b00-61be-4468-8b6d-e85603e33b67)\"" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7" podUID="42ac4b00-61be-4468-8b6d-e85603e33b67"
Jul 29 21:08:11.226585 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:11.226549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:08:11.227061 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:11.226668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:08:11.227341 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:11.227320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f314863c-51a2-4c85-aa0b-96f18234f702-service-ca-bundle\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:08:11.229119 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:11.229094    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f314863c-51a2-4c85-aa0b-96f18234f702-metrics-certs\") pod \"router-default-5dd67cb7f-4wd8g\" (UID: \"f314863c-51a2-4c85-aa0b-96f18234f702\") " pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:08:11.497067 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:11.496966    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jdq4f\""
Jul 29 21:08:11.504416 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:11.504394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:08:11.631856 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:11.631812    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5dd67cb7f-4wd8g"]
Jul 29 21:08:11.634017 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:08:11.633985    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf314863c_51a2_4c85_aa0b_96f18234f702.slice/crio-e7ab924a8dcdcbba8e7118524094b786c30cced7c5cf620bad50a69daa23c349 WatchSource:0}: Error finding container e7ab924a8dcdcbba8e7118524094b786c30cced7c5cf620bad50a69daa23c349: Status 404 returned error can't find the container with id e7ab924a8dcdcbba8e7118524094b786c30cced7c5cf620bad50a69daa23c349
Jul 29 21:08:12.570300 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:12.570197    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5dd67cb7f-4wd8g" event={"ID":"f314863c-51a2-4c85-aa0b-96f18234f702","Type":"ContainerStarted","Data":"a2e19e5b6c53940e7a500d63aa2f24051a2c92154be0e0cddc67a3ee543fb56e"}
Jul 29 21:08:12.570300 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:12.570258    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5dd67cb7f-4wd8g" event={"ID":"f314863c-51a2-4c85-aa0b-96f18234f702","Type":"ContainerStarted","Data":"e7ab924a8dcdcbba8e7118524094b786c30cced7c5cf620bad50a69daa23c349"}
Jul 29 21:08:12.571670 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:12.571644    2580 generic.go:356] "Generic (PLEG): container finished" podID="d154453e-9253-4b19-a42c-26c3a584b670" containerID="157ddb3436eafcf1a3b382560d7286097660d9d85c53ca6acb8bc4027a89b182" exitCode=255
Jul 29 21:08:12.571788 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:12.571684    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j" event={"ID":"d154453e-9253-4b19-a42c-26c3a584b670","Type":"ContainerDied","Data":"157ddb3436eafcf1a3b382560d7286097660d9d85c53ca6acb8bc4027a89b182"}
Jul 29 21:08:12.577443 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:12.577420    2580 scope.go:117] "RemoveContainer" containerID="157ddb3436eafcf1a3b382560d7286097660d9d85c53ca6acb8bc4027a89b182"
Jul 29 21:08:12.592546 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:12.592499    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5dd67cb7f-4wd8g" podStartSLOduration=33.592484465 podStartE2EDuration="33.592484465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:07:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:08:12.592092291 +0000 UTC m=+244.473366439" watchObservedRunningTime="2026-07-29 21:08:12.592484465 +0000 UTC m=+244.473758603"
Jul 29 21:08:13.504809 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:13.504771    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:08:13.507539 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:13.507511    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:08:13.576729 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:13.576692    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f8bc8f7b-f6k4j" event={"ID":"d154453e-9253-4b19-a42c-26c3a584b670","Type":"ContainerStarted","Data":"91b2bff4b1922b5781e9dc7716355d56949705a7ffda9cc647389af47078e3af"}
Jul 29 21:08:13.577229 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:13.576883    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:08:13.578085 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:13.578066    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5dd67cb7f-4wd8g"
Jul 29 21:08:15.159493 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:15.159457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:08:15.161718 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:15.161699    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/88e24522-aa6c-43bb-8805-b8e4d682e5a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j2scg\" (UID: \"88e24522-aa6c-43bb-8805-b8e4d682e5a2\") " pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:08:15.179623 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:15.179594    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qz6qg\""
Jul 29 21:08:15.187013 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:15.186992    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j2scg"
Jul 29 21:08:15.309632 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:15.309599    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j2scg"]
Jul 29 21:08:15.312787 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:08:15.312764    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88e24522_aa6c_43bb_8805_b8e4d682e5a2.slice/crio-b88d01a211e1a4391e1856eac2206a60b4f3ae9be3c944c9420f2b56b9bad6eb WatchSource:0}: Error finding container b88d01a211e1a4391e1856eac2206a60b4f3ae9be3c944c9420f2b56b9bad6eb: Status 404 returned error can't find the container with id b88d01a211e1a4391e1856eac2206a60b4f3ae9be3c944c9420f2b56b9bad6eb
Jul 29 21:08:15.584200 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:15.584115    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j2scg" event={"ID":"88e24522-aa6c-43bb-8805-b8e4d682e5a2","Type":"ContainerStarted","Data":"b88d01a211e1a4391e1856eac2206a60b4f3ae9be3c944c9420f2b56b9bad6eb"}
Jul 29 21:08:16.000580 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.000521    2580 patch_prober.go:28] interesting pod/image-registry-64b8979875-77628 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 29 21:08:16.000768 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.000593    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-64b8979875-77628" podUID="cdb47f20-85a3-485c-9b7d-acd9000f2a1a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 21:08:16.617948 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.617906    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"]
Jul 29 21:08:16.620810 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.620785    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:16.625582 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.625557    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 21:08:16.625999 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.625956    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 29 21:08:16.627108 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.627081    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 29 21:08:16.627230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.627177    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 21:08:16.627301 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.627253    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nb9df\""
Jul 29 21:08:16.633718 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.633694    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 21:08:16.642521 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.642497    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"]
Jul 29 21:08:16.773796 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.773756    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1731f9aa-3532-4995-bf66-c5f92137190d-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-rtxpt\" (UID: \"1731f9aa-3532-4995-bf66-c5f92137190d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:16.774004 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.773823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l59b\" (UniqueName: \"kubernetes.io/projected/1731f9aa-3532-4995-bf66-c5f92137190d-kube-api-access-6l59b\") pod \"prometheus-operator-84d69f794d-rtxpt\" (UID: \"1731f9aa-3532-4995-bf66-c5f92137190d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:16.774004 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.773919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1731f9aa-3532-4995-bf66-c5f92137190d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-rtxpt\" (UID: \"1731f9aa-3532-4995-bf66-c5f92137190d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:16.774105 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.774013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1731f9aa-3532-4995-bf66-c5f92137190d-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-rtxpt\" (UID: \"1731f9aa-3532-4995-bf66-c5f92137190d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:16.875503 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.875417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l59b\" (UniqueName: \"kubernetes.io/projected/1731f9aa-3532-4995-bf66-c5f92137190d-kube-api-access-6l59b\") pod \"prometheus-operator-84d69f794d-rtxpt\" (UID: \"1731f9aa-3532-4995-bf66-c5f92137190d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:16.875685 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.875507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1731f9aa-3532-4995-bf66-c5f92137190d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-rtxpt\" (UID: \"1731f9aa-3532-4995-bf66-c5f92137190d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:16.875685 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.875570    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1731f9aa-3532-4995-bf66-c5f92137190d-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-rtxpt\" (UID: \"1731f9aa-3532-4995-bf66-c5f92137190d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:16.875685 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.875607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1731f9aa-3532-4995-bf66-c5f92137190d-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-rtxpt\" (UID: \"1731f9aa-3532-4995-bf66-c5f92137190d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:16.876259 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.876228    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1731f9aa-3532-4995-bf66-c5f92137190d-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-rtxpt\" (UID: \"1731f9aa-3532-4995-bf66-c5f92137190d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:16.878247 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.878225    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1731f9aa-3532-4995-bf66-c5f92137190d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-rtxpt\" (UID: \"1731f9aa-3532-4995-bf66-c5f92137190d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:16.878247 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.878224    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1731f9aa-3532-4995-bf66-c5f92137190d-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-rtxpt\" (UID: \"1731f9aa-3532-4995-bf66-c5f92137190d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:16.885433 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.885400    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l59b\" (UniqueName: \"kubernetes.io/projected/1731f9aa-3532-4995-bf66-c5f92137190d-kube-api-access-6l59b\") pod \"prometheus-operator-84d69f794d-rtxpt\" (UID: \"1731f9aa-3532-4995-bf66-c5f92137190d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:16.933413 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:16.933379    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"
Jul 29 21:08:17.067419 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:17.067298    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-rtxpt"]
Jul 29 21:08:17.070313 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:08:17.070279    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1731f9aa_3532_4995_bf66_c5f92137190d.slice/crio-4084b4ba5416842b3a6f1587008de4db3eb60031421e0cc63371a512b3ee84d3 WatchSource:0}: Error finding container 4084b4ba5416842b3a6f1587008de4db3eb60031421e0cc63371a512b3ee84d3: Status 404 returned error can't find the container with id 4084b4ba5416842b3a6f1587008de4db3eb60031421e0cc63371a512b3ee84d3
Jul 29 21:08:17.593337 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:17.593288    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt" event={"ID":"1731f9aa-3532-4995-bf66-c5f92137190d","Type":"ContainerStarted","Data":"4084b4ba5416842b3a6f1587008de4db3eb60031421e0cc63371a512b3ee84d3"}
Jul 29 21:08:19.492874 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:19.492816    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:08:19.495393 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:19.495367    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b-metrics-certs\") pod \"network-metrics-daemon-6m4sz\" (UID: \"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b\") " pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:08:19.513420 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:19.513399    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-82zmz\""
Jul 29 21:08:19.520946 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:19.520922    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6m4sz"
Jul 29 21:08:21.145659 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:21.145628    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6m4sz"]
Jul 29 21:08:21.148247 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:08:21.148218    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce5211e6_dfbb_42bb_a7ee_a63bb3087a1b.slice/crio-cf9a1f26e3626c03161c549dfe5fa0e388e8adf1f83cceb31256e96d7986e85f WatchSource:0}: Error finding container cf9a1f26e3626c03161c549dfe5fa0e388e8adf1f83cceb31256e96d7986e85f: Status 404 returned error can't find the container with id cf9a1f26e3626c03161c549dfe5fa0e388e8adf1f83cceb31256e96d7986e85f
Jul 29 21:08:21.610851 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:21.610795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j2scg" event={"ID":"88e24522-aa6c-43bb-8805-b8e4d682e5a2","Type":"ContainerStarted","Data":"6f7ff77261600898500b04bb0cdc0c3d64cd4fb459e566ccb297ca0565581fdd"}
Jul 29 21:08:21.612494 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:21.612451    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6m4sz" event={"ID":"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b","Type":"ContainerStarted","Data":"cf9a1f26e3626c03161c549dfe5fa0e388e8adf1f83cceb31256e96d7986e85f"}
Jul 29 21:08:21.615202 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:21.615116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt" event={"ID":"1731f9aa-3532-4995-bf66-c5f92137190d","Type":"ContainerStarted","Data":"b5296c0e05567f98c2d8ffadf1ce0a0932bb596fd126c46d61f05d180e2dd985"}
Jul 29 21:08:21.615202 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:21.615166    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt" event={"ID":"1731f9aa-3532-4995-bf66-c5f92137190d","Type":"ContainerStarted","Data":"1c9c40f4d4d0583d2ad14f53a8f41856b425ed7ba89f60a48a2b222c11657bb6"}
Jul 29 21:08:21.633616 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:21.633551    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-rtxpt" podStartSLOduration=1.695019225 podStartE2EDuration="5.633531904s" totalImagesPullingTime="3.938512679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:08:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:08:21.63308967 +0000 UTC m=+253.514363830" watchObservedRunningTime="2026-07-29 21:08:21.633531904 +0000 UTC m=+253.514806043"
Jul 29 21:08:22.619956 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:22.619875    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6m4sz" event={"ID":"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b","Type":"ContainerStarted","Data":"df2f57005688c4471bb9f5f768dc19960fc172ece53998b48fb5aaab69e6ec38"}
Jul 29 21:08:22.620301 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:22.619977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6m4sz" event={"ID":"ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b","Type":"ContainerStarted","Data":"315f7e7802eb59dd634a4109463a8e50038cabf1940451252fdb11f4dbae352b"}
Jul 29 21:08:22.637442 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:22.637392    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6m4sz" podStartSLOduration=252.489982786 podStartE2EDuration="4m13.637372769s" totalImagesPullingTime="1.147389983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:08:22.63706684 +0000 UTC m=+254.518340981" watchObservedRunningTime="2026-07-29 21:08:22.637372769 +0000 UTC m=+254.518646909"
Jul 29 21:08:22.709749 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:22.709718    2580 scope.go:117] "RemoveContainer" containerID="409294e1d334fb5cc079a91c4a7138d88e28b9c5f6480a2e0cb562d12907b5d7"
Jul 29 21:08:22.709953 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:08:22.709922    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-ssvr7_openshift-console-operator(42ac4b00-61be-4468-8b6d-e85603e33b67)\"" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7" podUID="42ac4b00-61be-4468-8b6d-e85603e33b67"
Jul 29 21:08:24.000497 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.000412    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cwplq"]
Jul 29 21:08:24.007818 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.007789    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"]
Jul 29 21:08:24.007994 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.007951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.011491 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.011409    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 21:08:24.011491 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.011426    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 21:08:24.012869 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.012833    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.014006 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.013984    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cjm7d\""
Jul 29 21:08:24.014336 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.014318    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 21:08:24.016623 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.016053    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bqggx\""
Jul 29 21:08:24.016623 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.016110    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 29 21:08:24.016623 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.016250    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 29 21:08:24.016623 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.016348    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 29 21:08:24.020485 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.018928    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"]
Jul 29 21:08:24.138943 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.138908    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00abefd7-4870-4e91-b171-2dfe9b4db905-sys\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.139085 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.138950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkft8\" (UniqueName: \"kubernetes.io/projected/00abefd7-4870-4e91-b171-2dfe9b4db905-kube-api-access-lkft8\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.139085 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.138985    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.139085 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.139001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-989kw\" (UniqueName: \"kubernetes.io/projected/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-api-access-989kw\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.139085 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.139016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/00abefd7-4870-4e91-b171-2dfe9b4db905-root\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.139085 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.139067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.139289 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.139126    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00abefd7-4870-4e91-b171-2dfe9b4db905-metrics-client-ca\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.139289 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.139155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.139289 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.139179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.139289 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.139213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-tls\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.139289 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.139233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-accelerators-collector-config\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.139289 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.139258    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-textfile\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.139535 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.139297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.139535 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.139342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.139535 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.139418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-wtmp\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.240615 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.240579    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.240783 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.240632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-wtmp\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.240783 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.240668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00abefd7-4870-4e91-b171-2dfe9b4db905-sys\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.240783 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.240689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkft8\" (UniqueName: \"kubernetes.io/projected/00abefd7-4870-4e91-b171-2dfe9b4db905-kube-api-access-lkft8\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.240783 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:08:24.240709    2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 29 21:08:24.240783 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.240730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.240783 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.240757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-989kw\" (UniqueName: \"kubernetes.io/projected/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-api-access-989kw\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.240783 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.240774    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00abefd7-4870-4e91-b171-2dfe9b4db905-sys\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.241156 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:08:24.240785    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-tls podName:3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6 nodeName:}" failed. No retries permitted until 2026-07-29 21:08:24.740766063 +0000 UTC m=+256.622040194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-mqzbx" (UID: "3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6") : secret "kube-state-metrics-tls" not found
Jul 29 21:08:24.241156 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.240819    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-wtmp\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.241156 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.240869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/00abefd7-4870-4e91-b171-2dfe9b4db905-root\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.241156 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.240908    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.241156 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.240947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00abefd7-4870-4e91-b171-2dfe9b4db905-metrics-client-ca\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.241156 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.240972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.241156 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.240977    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/00abefd7-4870-4e91-b171-2dfe9b4db905-root\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.241156 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.241007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.241156 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.241049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-tls\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.241156 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.241076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-accelerators-collector-config\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.241631 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.241600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.241691 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.241655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00abefd7-4870-4e91-b171-2dfe9b4db905-metrics-client-ca\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.241691 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.241663    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.241784 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.241703    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-textfile\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.241784 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.241707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-accelerators-collector-config\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.241784 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.241717    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.241784 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.241753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.241998 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.241967    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-textfile\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.243731 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.243707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.244069 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.244053    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/00abefd7-4870-4e91-b171-2dfe9b4db905-node-exporter-tls\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.244214 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.244195    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.251404 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.251341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-989kw\" (UniqueName: \"kubernetes.io/projected/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-api-access-989kw\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.251404 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.251340    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkft8\" (UniqueName: \"kubernetes.io/projected/00abefd7-4870-4e91-b171-2dfe9b4db905-kube-api-access-lkft8\") pod \"node-exporter-cwplq\" (UID: \"00abefd7-4870-4e91-b171-2dfe9b4db905\") " pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.338488 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.338457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cwplq"
Jul 29 21:08:24.542828 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:08:24.542800    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00abefd7_4870_4e91_b171_2dfe9b4db905.slice/crio-452a20b019effc8222117c5f3221607ee3cd7e39808846410136cccced2d9c14 WatchSource:0}: Error finding container 452a20b019effc8222117c5f3221607ee3cd7e39808846410136cccced2d9c14: Status 404 returned error can't find the container with id 452a20b019effc8222117c5f3221607ee3cd7e39808846410136cccced2d9c14
Jul 29 21:08:24.627586 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.627554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cwplq" event={"ID":"00abefd7-4870-4e91-b171-2dfe9b4db905","Type":"ContainerStarted","Data":"452a20b019effc8222117c5f3221607ee3cd7e39808846410136cccced2d9c14"}
Jul 29 21:08:24.746157 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:24.746128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:24.746315 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:08:24.746275    2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 29 21:08:24.746359 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:08:24.746331    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-tls podName:3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6 nodeName:}" failed. No retries permitted until 2026-07-29 21:08:25.7463164 +0000 UTC m=+257.627590516 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-mqzbx" (UID: "3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6") : secret "kube-state-metrics-tls" not found
Jul 29 21:08:25.633470 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:25.633441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j2scg" event={"ID":"88e24522-aa6c-43bb-8805-b8e4d682e5a2","Type":"ContainerStarted","Data":"f24d631cd95fa46482734930bd81822988fed1f6f09b0901a88cff77ac4dee92"}
Jul 29 21:08:25.635157 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:25.635131    2580 generic.go:356] "Generic (PLEG): container finished" podID="00abefd7-4870-4e91-b171-2dfe9b4db905" containerID="a1341a559d32ee446fda885194286de6618a7b07f6e6becb20ddbe4cffb30015" exitCode=0
Jul 29 21:08:25.635282 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:25.635206    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cwplq" event={"ID":"00abefd7-4870-4e91-b171-2dfe9b4db905","Type":"ContainerDied","Data":"a1341a559d32ee446fda885194286de6618a7b07f6e6becb20ddbe4cffb30015"}
Jul 29 21:08:25.756192 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:25.756088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:25.758691 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:25.758661    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-mqzbx\" (UID: \"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:25.853012 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:25.852971    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"
Jul 29 21:08:25.997696 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:25.997498    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx"]
Jul 29 21:08:25.998807 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:25.998737    2580 patch_prober.go:28] interesting pod/image-registry-64b8979875-77628 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 29 21:08:25.998807 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:25.998788    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-64b8979875-77628" podUID="cdb47f20-85a3-485c-9b7d-acd9000f2a1a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 21:08:25.999875 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:08:25.999825    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dd3f78f_08fe_4f4d_a3ed_39d3eead0aa6.slice/crio-aadaf9d47a0cf537b2833ba877398799fe79bd194cf640d1a1db26c82ad7f061 WatchSource:0}: Error finding container aadaf9d47a0cf537b2833ba877398799fe79bd194cf640d1a1db26c82ad7f061: Status 404 returned error can't find the container with id aadaf9d47a0cf537b2833ba877398799fe79bd194cf640d1a1db26c82ad7f061
Jul 29 21:08:26.640599 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.640558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx" event={"ID":"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6","Type":"ContainerStarted","Data":"aadaf9d47a0cf537b2833ba877398799fe79bd194cf640d1a1db26c82ad7f061"}
Jul 29 21:08:26.642928 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.642897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cwplq" event={"ID":"00abefd7-4870-4e91-b171-2dfe9b4db905","Type":"ContainerStarted","Data":"b4514b9c82c62731d7261a46d55f17d3a0e4bfa5e41e41deaaea53d5b9007a90"}
Jul 29 21:08:26.643061 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.642953    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cwplq" event={"ID":"00abefd7-4870-4e91-b171-2dfe9b4db905","Type":"ContainerStarted","Data":"76352e22ca41ad7c9d9735ae4a1b8598e50f6bed4c09af3ce9dd8407763bc1ac"}
Jul 29 21:08:26.667036 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.666993    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cwplq" podStartSLOduration=2.6764541189999997 podStartE2EDuration="3.666978432s" totalImagesPullingTime="917.591771ms" totalInitContainerRuntime="72.932542ms" isStatefulPod=true podCreationTimestamp="2026-07-29 21:08:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:08:26.66336649 +0000 UTC m=+258.544640627" watchObservedRunningTime="2026-07-29 21:08:26.666978432 +0000 UTC m=+258.548252548"
Jul 29 21:08:26.947560 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.947486    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-74df9db688-f58fb"]
Jul 29 21:08:26.955906 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.955884    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:26.958765 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.958733    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 29 21:08:26.958885 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.958771    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-5j5ml\""
Jul 29 21:08:26.958947 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.958892    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 29 21:08:26.958947 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.958742    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 29 21:08:26.959210 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.959189    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 29 21:08:26.959311 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.959281    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 29 21:08:26.959379 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.959325    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-73v7bn1j1jpjf\""
Jul 29 21:08:26.962102 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.962083    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-74df9db688-f58fb"]
Jul 29 21:08:26.968166 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.968141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-metrics-client-ca\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:26.968279 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.968187    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-tls\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:26.968279 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.968221    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:26.968279 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.968263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:26.968424 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.968294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:26.968424 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.968318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-grpc-tls\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:26.968524 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.968460    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j75l\" (UniqueName: \"kubernetes.io/projected/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-kube-api-access-8j75l\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:26.968571 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:26.968534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.069035 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.068999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j75l\" (UniqueName: \"kubernetes.io/projected/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-kube-api-access-8j75l\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.069230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.069066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.069230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.069131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-metrics-client-ca\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.069230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.069156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-tls\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.069230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.069179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.069230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.069208    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.069230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.069235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.069592 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.069251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-grpc-tls\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.070519 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.070481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-metrics-client-ca\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.072727 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.072688    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.072855 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.072753    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.073207 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.073173    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-tls\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.073291 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.073244    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.073360 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.073295    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-grpc-tls\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.073594 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.073570    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.078001 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.077982    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j75l\" (UniqueName: \"kubernetes.io/projected/c9e6fcbb-ee46-47d2-a409-e4748c66ce90-kube-api-access-8j75l\") pod \"thanos-querier-74df9db688-f58fb\" (UID: \"c9e6fcbb-ee46-47d2-a409-e4748c66ce90\") " pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:27.267484 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:27.267386    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:28.281864 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.281828    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"]
Jul 29 21:08:28.286325 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.286304    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.290462 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.290403    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 21:08:28.290462 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.290432    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-56pb7b677acg8\""
Jul 29 21:08:28.290462 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.290459    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 29 21:08:28.290691 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.290511    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jtnbp\""
Jul 29 21:08:28.290780 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.290758    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 29 21:08:28.290892 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.290818    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 29 21:08:28.294825 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.294803    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"]
Jul 29 21:08:28.300639 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.300619    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-74df9db688-f58fb"]
Jul 29 21:08:28.305007 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:08:28.304981    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9e6fcbb_ee46_47d2_a409_e4748c66ce90.slice/crio-e2705f3548af7018884f3bac8ba301920faf122f6ee8a4464c0b05549d6aee37 WatchSource:0}: Error finding container e2705f3548af7018884f3bac8ba301920faf122f6ee8a4464c0b05549d6aee37: Status 404 returned error can't find the container with id e2705f3548af7018884f3bac8ba301920faf122f6ee8a4464c0b05549d6aee37
Jul 29 21:08:28.381107 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.381078    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fe1085ac-7489-457c-aa7c-cffe927bf9b1-metrics-server-audit-profiles\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.381239 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.381115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fe1085ac-7489-457c-aa7c-cffe927bf9b1-secret-metrics-server-tls\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.381239 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.381158    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fe1085ac-7489-457c-aa7c-cffe927bf9b1-secret-metrics-server-client-certs\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.381239 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.381211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fe1085ac-7489-457c-aa7c-cffe927bf9b1-audit-log\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.381372 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.381240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psq8j\" (UniqueName: \"kubernetes.io/projected/fe1085ac-7489-457c-aa7c-cffe927bf9b1-kube-api-access-psq8j\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.381372 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.381338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe1085ac-7489-457c-aa7c-cffe927bf9b1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.381372 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.381365    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fe1085ac-7489-457c-aa7c-cffe927bf9b1-client-ca-bundle\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.481869 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.481769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe1085ac-7489-457c-aa7c-cffe927bf9b1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.481869 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.481815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fe1085ac-7489-457c-aa7c-cffe927bf9b1-client-ca-bundle\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.482122 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.481884    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fe1085ac-7489-457c-aa7c-cffe927bf9b1-metrics-server-audit-profiles\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.482122 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.481914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fe1085ac-7489-457c-aa7c-cffe927bf9b1-secret-metrics-server-tls\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.482122 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.481979    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fe1085ac-7489-457c-aa7c-cffe927bf9b1-secret-metrics-server-client-certs\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.482122 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.482010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fe1085ac-7489-457c-aa7c-cffe927bf9b1-audit-log\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.482122 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.482047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psq8j\" (UniqueName: \"kubernetes.io/projected/fe1085ac-7489-457c-aa7c-cffe927bf9b1-kube-api-access-psq8j\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.482547 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.482495    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fe1085ac-7489-457c-aa7c-cffe927bf9b1-audit-log\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.482747 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.482721    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe1085ac-7489-457c-aa7c-cffe927bf9b1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.483087 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.483029    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fe1085ac-7489-457c-aa7c-cffe927bf9b1-metrics-server-audit-profiles\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.485237 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.485190    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fe1085ac-7489-457c-aa7c-cffe927bf9b1-secret-metrics-server-tls\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.485237 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.485190    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fe1085ac-7489-457c-aa7c-cffe927bf9b1-client-ca-bundle\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.485382 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.485244    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fe1085ac-7489-457c-aa7c-cffe927bf9b1-secret-metrics-server-client-certs\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.490217 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.490194    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psq8j\" (UniqueName: \"kubernetes.io/projected/fe1085ac-7489-457c-aa7c-cffe927bf9b1-kube-api-access-psq8j\") pod \"metrics-server-6bcfbcc6cc-mb5zt\" (UID: \"fe1085ac-7489-457c-aa7c-cffe927bf9b1\") " pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.600494 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.600461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:28.652686 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.652644    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74df9db688-f58fb" event={"ID":"c9e6fcbb-ee46-47d2-a409-e4748c66ce90","Type":"ContainerStarted","Data":"e2705f3548af7018884f3bac8ba301920faf122f6ee8a4464c0b05549d6aee37"}
Jul 29 21:08:28.655212 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.655181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j2scg" event={"ID":"88e24522-aa6c-43bb-8805-b8e4d682e5a2","Type":"ContainerStarted","Data":"e5a2e129fdd76e936a243e51a9c9c99538c7fe5c74e3824d02b94a7e740c66d9"}
Jul 29 21:08:28.673976 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.673917    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-j2scg" podStartSLOduration=32.914974651 podStartE2EDuration="45.673892812s" totalImagesPullingTime="12.758918161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:07:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:08:28.673749733 +0000 UTC m=+260.555023872" watchObservedRunningTime="2026-07-29 21:08:28.673892812 +0000 UTC m=+260.555166951"
Jul 29 21:08:28.762164 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:28.760816    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"]
Jul 29 21:08:28.766344 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:08:28.766308    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe1085ac_7489_457c_aa7c_cffe927bf9b1.slice/crio-2a2330c255194bd3236a64004c8cb89ae9746aae67374d974a1c9823bf4573a9 WatchSource:0}: Error finding container 2a2330c255194bd3236a64004c8cb89ae9746aae67374d974a1c9823bf4573a9: Status 404 returned error can't find the container with id 2a2330c255194bd3236a64004c8cb89ae9746aae67374d974a1c9823bf4573a9
Jul 29 21:08:29.666448 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:29.663134    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx" event={"ID":"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6","Type":"ContainerStarted","Data":"ed16ded3f6c6ddfc7cb90a9c413d8b37e7ac6ff918083995dc150ed8c7a40dee"}
Jul 29 21:08:29.666448 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:29.663236    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx" event={"ID":"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6","Type":"ContainerStarted","Data":"1015db15550039627effaf49365a0348127b1f32da26874217ef7d8d3d4f3081"}
Jul 29 21:08:29.666448 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:29.663273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx" event={"ID":"3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6","Type":"ContainerStarted","Data":"61fea5e1f8361edadfeff1842cac1d2b54df649e6f2ed1f0a0b62df7d57cd69f"}
Jul 29 21:08:29.667865 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:29.667807    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt" event={"ID":"fe1085ac-7489-457c-aa7c-cffe927bf9b1","Type":"ContainerStarted","Data":"2a2330c255194bd3236a64004c8cb89ae9746aae67374d974a1c9823bf4573a9"}
Jul 29 21:08:31.011246 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.011200    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64b8979875-77628" podUID="cdb47f20-85a3-485c-9b7d-acd9000f2a1a" containerName="registry" containerID="cri-o://4909f042124defc1d18b7bc9f3eef5d067aee7a8266e0d56d5098d9024841aba" gracePeriod=30
Jul 29 21:08:31.257261 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.257230    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:08:31.283020 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.282976    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mqzbx" podStartSLOduration=5.093134485 podStartE2EDuration="8.282961971s" totalImagesPullingTime="3.189827486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:08:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:08:29.685933118 +0000 UTC m=+261.567207257" watchObservedRunningTime="2026-07-29 21:08:31.282961971 +0000 UTC m=+263.164236109"
Jul 29 21:08:31.313024 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.312996    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-image-registry-private-configuration\") pod \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") "
Jul 29 21:08:31.313103 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.313041    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-ca-trust-extracted\") pod \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") "
Jul 29 21:08:31.313103 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.313070    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-installation-pull-secrets\") pod \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") "
Jul 29 21:08:31.313165 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.313103    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-certificates\") pod \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") "
Jul 29 21:08:31.313195 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.313169    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-bound-sa-token\") pod \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") "
Jul 29 21:08:31.313237 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.313196    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls\") pod \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") "
Jul 29 21:08:31.313237 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.313231    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-trusted-ca\") pod \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") "
Jul 29 21:08:31.313330 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.313293    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhpjw\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-kube-api-access-qhpjw\") pod \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\" (UID: \"cdb47f20-85a3-485c-9b7d-acd9000f2a1a\") "
Jul 29 21:08:31.313677 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.313649    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cdb47f20-85a3-485c-9b7d-acd9000f2a1a" (UID: "cdb47f20-85a3-485c-9b7d-acd9000f2a1a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:08:31.314156 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.314132    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cdb47f20-85a3-485c-9b7d-acd9000f2a1a" (UID: "cdb47f20-85a3-485c-9b7d-acd9000f2a1a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:08:31.315917 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.315876    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cdb47f20-85a3-485c-9b7d-acd9000f2a1a" (UID: "cdb47f20-85a3-485c-9b7d-acd9000f2a1a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:08:31.316094 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.316068    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cdb47f20-85a3-485c-9b7d-acd9000f2a1a" (UID: "cdb47f20-85a3-485c-9b7d-acd9000f2a1a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:08:31.316186 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.316165    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cdb47f20-85a3-485c-9b7d-acd9000f2a1a" (UID: "cdb47f20-85a3-485c-9b7d-acd9000f2a1a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:08:31.316554 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.316523    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cdb47f20-85a3-485c-9b7d-acd9000f2a1a" (UID: "cdb47f20-85a3-485c-9b7d-acd9000f2a1a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:08:31.317203 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.317175    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-kube-api-access-qhpjw" (OuterVolumeSpecName: "kube-api-access-qhpjw") pod "cdb47f20-85a3-485c-9b7d-acd9000f2a1a" (UID: "cdb47f20-85a3-485c-9b7d-acd9000f2a1a"). InnerVolumeSpecName "kube-api-access-qhpjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:08:31.324130 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.324110    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cdb47f20-85a3-485c-9b7d-acd9000f2a1a" (UID: "cdb47f20-85a3-485c-9b7d-acd9000f2a1a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:08:31.414879 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.414826    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-image-registry-private-configuration\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:08:31.414879 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.414878    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-ca-trust-extracted\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:08:31.415077 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.414890    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-installation-pull-secrets\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:08:31.415077 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.414899    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-certificates\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:08:31.415077 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.414908    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-bound-sa-token\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:08:31.415077 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.414916    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-registry-tls\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:08:31.415077 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.414923    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-trusted-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:08:31.415077 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.414932    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhpjw\" (UniqueName: \"kubernetes.io/projected/cdb47f20-85a3-485c-9b7d-acd9000f2a1a-kube-api-access-qhpjw\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:08:31.679862 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.679814    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74df9db688-f58fb" event={"ID":"c9e6fcbb-ee46-47d2-a409-e4748c66ce90","Type":"ContainerStarted","Data":"5024ee0c798b156251c662b664d6e5002946075a5aec3fbba5dd9d6840ab2b1a"}
Jul 29 21:08:31.680005 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.679914    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74df9db688-f58fb" event={"ID":"c9e6fcbb-ee46-47d2-a409-e4748c66ce90","Type":"ContainerStarted","Data":"1e220ac1c2b229e90401dfea54d9c3fdc201983f8d96f4712e32caf72beda0b2"}
Jul 29 21:08:31.680005 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.679946    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74df9db688-f58fb" event={"ID":"c9e6fcbb-ee46-47d2-a409-e4748c66ce90","Type":"ContainerStarted","Data":"c2f360cf373d3727d6b3ad40c8b7ca8592381b7c6dc9a436f560bfaa1791a623"}
Jul 29 21:08:31.680906 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.680885    2580 generic.go:356] "Generic (PLEG): container finished" podID="cdb47f20-85a3-485c-9b7d-acd9000f2a1a" containerID="4909f042124defc1d18b7bc9f3eef5d067aee7a8266e0d56d5098d9024841aba" exitCode=0
Jul 29 21:08:31.681001 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.680932    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64b8979875-77628"
Jul 29 21:08:31.681001 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.680953    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b8979875-77628" event={"ID":"cdb47f20-85a3-485c-9b7d-acd9000f2a1a","Type":"ContainerDied","Data":"4909f042124defc1d18b7bc9f3eef5d067aee7a8266e0d56d5098d9024841aba"}
Jul 29 21:08:31.681001 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.680997    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64b8979875-77628" event={"ID":"cdb47f20-85a3-485c-9b7d-acd9000f2a1a","Type":"ContainerDied","Data":"6bf8120de1b8c7d9f5d5292ece3875a06d8877da587f814bcab646274d76ed9d"}
Jul 29 21:08:31.681157 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.681021    2580 scope.go:117] "RemoveContainer" containerID="4909f042124defc1d18b7bc9f3eef5d067aee7a8266e0d56d5098d9024841aba"
Jul 29 21:08:31.682573 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.682552    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt" event={"ID":"fe1085ac-7489-457c-aa7c-cffe927bf9b1","Type":"ContainerStarted","Data":"7b9e40d7d7690373b397d8cf314ff4571abd5c7fe99b97e68270f90b444b18f5"}
Jul 29 21:08:31.689855 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.689619    2580 scope.go:117] "RemoveContainer" containerID="4909f042124defc1d18b7bc9f3eef5d067aee7a8266e0d56d5098d9024841aba"
Jul 29 21:08:31.690006 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:08:31.689978    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4909f042124defc1d18b7bc9f3eef5d067aee7a8266e0d56d5098d9024841aba\": container with ID starting with 4909f042124defc1d18b7bc9f3eef5d067aee7a8266e0d56d5098d9024841aba not found: ID does not exist" containerID="4909f042124defc1d18b7bc9f3eef5d067aee7a8266e0d56d5098d9024841aba"
Jul 29 21:08:31.690066 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.690027    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4909f042124defc1d18b7bc9f3eef5d067aee7a8266e0d56d5098d9024841aba"} err="failed to get container status \"4909f042124defc1d18b7bc9f3eef5d067aee7a8266e0d56d5098d9024841aba\": rpc error: code = NotFound desc = could not find container \"4909f042124defc1d18b7bc9f3eef5d067aee7a8266e0d56d5098d9024841aba\": container with ID starting with 4909f042124defc1d18b7bc9f3eef5d067aee7a8266e0d56d5098d9024841aba not found: ID does not exist"
Jul 29 21:08:31.702467 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.702421    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt" podStartSLOduration=1.374699775 podStartE2EDuration="3.702410498s" totalImagesPullingTime="2.327710723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:08:31.701625497 +0000 UTC m=+263.582899632" watchObservedRunningTime="2026-07-29 21:08:31.702410498 +0000 UTC m=+263.583684614"
Jul 29 21:08:31.715628 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.715600    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64b8979875-77628"]
Jul 29 21:08:31.721629 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:31.721609    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64b8979875-77628"]
Jul 29 21:08:32.689244 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:32.689209    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74df9db688-f58fb" event={"ID":"c9e6fcbb-ee46-47d2-a409-e4748c66ce90","Type":"ContainerStarted","Data":"4e473131d37f34bb8d7d3d913772243535c9c88736842d00e9c7c1826540d2f8"}
Jul 29 21:08:32.689675 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:32.689319    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74df9db688-f58fb" event={"ID":"c9e6fcbb-ee46-47d2-a409-e4748c66ce90","Type":"ContainerStarted","Data":"4684521a2b6ec4bf3d6e1946932960ac7b29c58c51a8758d0bd7d477f789af61"}
Jul 29 21:08:32.689675 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:32.689363    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:32.689675 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:32.689394    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74df9db688-f58fb" event={"ID":"c9e6fcbb-ee46-47d2-a409-e4748c66ce90","Type":"ContainerStarted","Data":"9db77593dd15c07ec803b5bcf52a12eaf99e929336e4936e12c89d1338b12bd3"}
Jul 29 21:08:32.714139 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:32.714093    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-74df9db688-f58fb" podStartSLOduration=2.951378152 podStartE2EDuration="6.714077529s" totalImagesPullingTime="3.762699377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:08:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:08:32.712151191 +0000 UTC m=+264.593425330" watchObservedRunningTime="2026-07-29 21:08:32.714077529 +0000 UTC m=+264.595351665"
Jul 29 21:08:32.715463 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:32.715441    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdb47f20-85a3-485c-9b7d-acd9000f2a1a" path="/var/lib/kubelet/pods/cdb47f20-85a3-485c-9b7d-acd9000f2a1a/volumes"
Jul 29 21:08:35.709523 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:35.709486    2580 scope.go:117] "RemoveContainer" containerID="409294e1d334fb5cc079a91c4a7138d88e28b9c5f6480a2e0cb562d12907b5d7"
Jul 29 21:08:36.571786 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.571751    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-jrpkc"]
Jul 29 21:08:36.572111 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.572097    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdb47f20-85a3-485c-9b7d-acd9000f2a1a" containerName="registry"
Jul 29 21:08:36.572165 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.572114    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdb47f20-85a3-485c-9b7d-acd9000f2a1a" containerName="registry"
Jul 29 21:08:36.574164 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.572792    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdb47f20-85a3-485c-9b7d-acd9000f2a1a" containerName="registry"
Jul 29 21:08:36.576890 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.576496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-jrpkc"
Jul 29 21:08:36.579603 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.579580    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-xgrtf\""
Jul 29 21:08:36.580129 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.580105    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 21:08:36.580781 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.580763    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 21:08:36.585626 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.585604    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-jrpkc"]
Jul 29 21:08:36.660014 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.659988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjs2s\" (UniqueName: \"kubernetes.io/projected/3433dc8c-6a78-4170-8c53-48d19c2ff6d3-kube-api-access-jjs2s\") pod \"downloads-7c97dc4555-jrpkc\" (UID: \"3433dc8c-6a78-4170-8c53-48d19c2ff6d3\") " pod="openshift-console/downloads-7c97dc4555-jrpkc"
Jul 29 21:08:36.708773 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.708754    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:08:36.712484 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.712460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7" event={"ID":"42ac4b00-61be-4468-8b6d-e85603e33b67","Type":"ContainerStarted","Data":"dc843125b59c995f98066e4e2e31f328732e10ae163601569c6c0b9f1e5029f1"}
Jul 29 21:08:36.712795 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.712728    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:08:36.717522 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.717502    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7"
Jul 29 21:08:36.732134 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.732090    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-ssvr7" podStartSLOduration=54.695982433 podStartE2EDuration="57.732074011s" totalImagesPullingTime="3.036091578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:07:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:08:36.731916809 +0000 UTC m=+268.613190951" watchObservedRunningTime="2026-07-29 21:08:36.732074011 +0000 UTC m=+268.613348150"
Jul 29 21:08:36.760826 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.760804    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjs2s\" (UniqueName: \"kubernetes.io/projected/3433dc8c-6a78-4170-8c53-48d19c2ff6d3-kube-api-access-jjs2s\") pod \"downloads-7c97dc4555-jrpkc\" (UID: \"3433dc8c-6a78-4170-8c53-48d19c2ff6d3\") " pod="openshift-console/downloads-7c97dc4555-jrpkc"
Jul 29 21:08:36.783128 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.783102    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjs2s\" (UniqueName: \"kubernetes.io/projected/3433dc8c-6a78-4170-8c53-48d19c2ff6d3-kube-api-access-jjs2s\") pod \"downloads-7c97dc4555-jrpkc\" (UID: \"3433dc8c-6a78-4170-8c53-48d19c2ff6d3\") " pod="openshift-console/downloads-7c97dc4555-jrpkc"
Jul 29 21:08:36.887184 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:36.887156    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-jrpkc"
Jul 29 21:08:37.009588 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:37.009554    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-jrpkc"]
Jul 29 21:08:37.013974 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:08:37.013940    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3433dc8c_6a78_4170_8c53_48d19c2ff6d3.slice/crio-fab43bcf28c4ecddda06682a1fd5a05719514f2059edadb2daa64eedb0240c64 WatchSource:0}: Error finding container fab43bcf28c4ecddda06682a1fd5a05719514f2059edadb2daa64eedb0240c64: Status 404 returned error can't find the container with id fab43bcf28c4ecddda06682a1fd5a05719514f2059edadb2daa64eedb0240c64
Jul 29 21:08:37.714377 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:37.714334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-jrpkc" event={"ID":"3433dc8c-6a78-4170-8c53-48d19c2ff6d3","Type":"ContainerStarted","Data":"fab43bcf28c4ecddda06682a1fd5a05719514f2059edadb2daa64eedb0240c64"}
Jul 29 21:08:38.701225 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:38.701195    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-74df9db688-f58fb"
Jul 29 21:08:46.976082 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:46.976050    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-895466b4b-h9z7h"]
Jul 29 21:08:47.176412 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.176374    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-895466b4b-h9z7h"]
Jul 29 21:08:47.176592 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.176503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.179225 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.179203    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-64d8z\""
Jul 29 21:08:47.179225 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.179219    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 21:08:47.179418 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.179361    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 21:08:47.179505 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.179483    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 21:08:47.180587 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.180559    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 21:08:47.180670 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.180592    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 21:08:47.253640 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.253605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhpxk\" (UniqueName: \"kubernetes.io/projected/bd830671-514c-48d4-bd0e-b69694f7007d-kube-api-access-hhpxk\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.253761 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.253654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-service-ca\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.253761 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.253678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-oauth-serving-cert\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.253761 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.253743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-console-config\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.253931 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.253800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd830671-514c-48d4-bd0e-b69694f7007d-console-oauth-config\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.253931 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.253820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd830671-514c-48d4-bd0e-b69694f7007d-console-serving-cert\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.354486 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.354458    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd830671-514c-48d4-bd0e-b69694f7007d-console-oauth-config\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.354609 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.354489    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd830671-514c-48d4-bd0e-b69694f7007d-console-serving-cert\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.354609 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.354550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhpxk\" (UniqueName: \"kubernetes.io/projected/bd830671-514c-48d4-bd0e-b69694f7007d-kube-api-access-hhpxk\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.354609 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.354571    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-service-ca\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.354609 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.354586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-oauth-serving-cert\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.354767 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.354609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-console-config\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.355310 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.355285    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-service-ca\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.355427 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.355401    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-oauth-serving-cert\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.355492 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.355401    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-console-config\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.356991 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.356970    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd830671-514c-48d4-bd0e-b69694f7007d-console-oauth-config\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.357114 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.357055    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd830671-514c-48d4-bd0e-b69694f7007d-console-serving-cert\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.362853 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.362820    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhpxk\" (UniqueName: \"kubernetes.io/projected/bd830671-514c-48d4-bd0e-b69694f7007d-kube-api-access-hhpxk\") pod \"console-895466b4b-h9z7h\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") " pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:47.487963 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:47.487879    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:08:48.327121 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:08:48.326298    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7hv9z" podUID="17ed28ff-e083-4d5e-819c-142ff66b85dd"
Jul 29 21:08:48.600652 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:48.600573    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:48.600825 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:48.600669    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:08:48.760622 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:48.760587    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:08:51.494330 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:51.494292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:08:51.496938 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:51.496909    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17ed28ff-e083-4d5e-819c-142ff66b85dd-metrics-tls\") pod \"dns-default-7hv9z\" (UID: \"17ed28ff-e083-4d5e-819c-142ff66b85dd\") " pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:08:51.764810 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:51.764731    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5hxhk\""
Jul 29 21:08:51.772492 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:51.772439    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:08:54.433749 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:54.433725    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7hv9z"]
Jul 29 21:08:54.436272 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:08:54.436240    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ed28ff_e083_4d5e_819c_142ff66b85dd.slice/crio-04aaa4e402d20c22534a88a9ca2b1051d0a06a586937ad014c3e5398a6a1dad8 WatchSource:0}: Error finding container 04aaa4e402d20c22534a88a9ca2b1051d0a06a586937ad014c3e5398a6a1dad8: Status 404 returned error can't find the container with id 04aaa4e402d20c22534a88a9ca2b1051d0a06a586937ad014c3e5398a6a1dad8
Jul 29 21:08:54.450735 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:54.450709    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-895466b4b-h9z7h"]
Jul 29 21:08:54.455833 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:08:54.455810    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd830671_514c_48d4_bd0e_b69694f7007d.slice/crio-460206b49fe1d63424f9217281ec8f1e4221b8d79bae40bfa442f93323bb0f87 WatchSource:0}: Error finding container 460206b49fe1d63424f9217281ec8f1e4221b8d79bae40bfa442f93323bb0f87: Status 404 returned error can't find the container with id 460206b49fe1d63424f9217281ec8f1e4221b8d79bae40bfa442f93323bb0f87
Jul 29 21:08:54.789189 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:54.789148    2580 generic.go:356] "Generic (PLEG): container finished" podID="ffee65c3-a94f-436b-907e-f5c81f28752f" containerID="b04621ad5718e2a13874795e22e8fa50542bf00b5bbe832c0fa08ac5968c8495" exitCode=0
Jul 29 21:08:54.789375 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:54.789226    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h" event={"ID":"ffee65c3-a94f-436b-907e-f5c81f28752f","Type":"ContainerDied","Data":"b04621ad5718e2a13874795e22e8fa50542bf00b5bbe832c0fa08ac5968c8495"}
Jul 29 21:08:54.789791 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:54.789670    2580 scope.go:117] "RemoveContainer" containerID="b04621ad5718e2a13874795e22e8fa50542bf00b5bbe832c0fa08ac5968c8495"
Jul 29 21:08:54.791241 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:54.791111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-jrpkc" event={"ID":"3433dc8c-6a78-4170-8c53-48d19c2ff6d3","Type":"ContainerStarted","Data":"72fcd769e17cdd0d35248eab5003c3e7e1fa922975d7bc58be33a9eb1cda0cc8"}
Jul 29 21:08:54.791379 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:54.791360    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-jrpkc"
Jul 29 21:08:54.793213 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:54.793190    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7hv9z" event={"ID":"17ed28ff-e083-4d5e-819c-142ff66b85dd","Type":"ContainerStarted","Data":"04aaa4e402d20c22534a88a9ca2b1051d0a06a586937ad014c3e5398a6a1dad8"}
Jul 29 21:08:54.795058 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:54.795038    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-895466b4b-h9z7h" event={"ID":"bd830671-514c-48d4-bd0e-b69694f7007d","Type":"ContainerStarted","Data":"460206b49fe1d63424f9217281ec8f1e4221b8d79bae40bfa442f93323bb0f87"}
Jul 29 21:08:54.802212 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:54.802186    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-jrpkc"
Jul 29 21:08:54.827100 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:54.827045    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-jrpkc" podStartSLOduration=1.458434053 podStartE2EDuration="18.827023332s" totalImagesPullingTime="17.368589279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:08:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:08:54.824998336 +0000 UTC m=+286.706272498" watchObservedRunningTime="2026-07-29 21:08:54.827023332 +0000 UTC m=+286.708297471"
Jul 29 21:08:55.804367 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:55.804314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-h7x8h" event={"ID":"ffee65c3-a94f-436b-907e-f5c81f28752f","Type":"ContainerStarted","Data":"54a266041fe447c161e32c37550acae5554c194467272b8677046c4b3ec6e305"}
Jul 29 21:08:59.827552 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:59.827451    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7hv9z" event={"ID":"17ed28ff-e083-4d5e-819c-142ff66b85dd","Type":"ContainerStarted","Data":"7192e37c24087e4e195ca1bc94b504ee738939317c49e0e8cdf097946f4fa1fb"}
Jul 29 21:08:59.827552 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:59.827521    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7hv9z" event={"ID":"17ed28ff-e083-4d5e-819c-142ff66b85dd","Type":"ContainerStarted","Data":"99212932490aa72943012c40a63ed9573ab27e352512c3b7e76dd2b5a932b2a9"}
Jul 29 21:08:59.827552 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:59.827553    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:08:59.829427 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:59.829391    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-895466b4b-h9z7h" event={"ID":"bd830671-514c-48d4-bd0e-b69694f7007d","Type":"ContainerStarted","Data":"e27f25988bbced3d044f323c0f6c836a1a2593f7a600e7227e09cd7f80faca48"}
Jul 29 21:08:59.845432 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:59.845379    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7hv9z" podStartSLOduration=254.175508408 podStartE2EDuration="4m18.845359635s" totalImagesPullingTime="4.669851227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:04:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:08:59.844347432 +0000 UTC m=+291.725621594" watchObservedRunningTime="2026-07-29 21:08:59.845359635 +0000 UTC m=+291.726633774"
Jul 29 21:08:59.864242 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:08:59.864186    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-895466b4b-h9z7h" podStartSLOduration=9.206983797 podStartE2EDuration="13.864165165s" totalImagesPullingTime="4.657181368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:08:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:08:59.863348211 +0000 UTC m=+291.744622351" watchObservedRunningTime="2026-07-29 21:08:59.864165165 +0000 UTC m=+291.745439304"
Jul 29 21:09:06.386256 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:06.386224    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-895466b4b-h9z7h"]
Jul 29 21:09:07.488895 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:07.488852    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:09:08.498399 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:08.498375    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:09:08.498752 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:08.498591    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:09:08.502807 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:08.502787    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:09:08.502941 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:08.502892    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:09:08.510115 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:08.510098    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 21:09:08.606977 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:08.606953    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:09:08.611313 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:08.611286    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6bcfbcc6cc-mb5zt"
Jul 29 21:09:09.013293 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:08.868831    2580 generic.go:356] "Generic (PLEG): container finished" podID="d135c4fb-0ebd-485a-8dbb-39369e431db7" containerID="e3063de80d0ecae28ba7c3bc2bcec164db61c130a2f71e626fb072427f098f52" exitCode=0
Jul 29 21:09:09.013293 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:08.868875    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg" event={"ID":"d135c4fb-0ebd-485a-8dbb-39369e431db7","Type":"ContainerDied","Data":"e3063de80d0ecae28ba7c3bc2bcec164db61c130a2f71e626fb072427f098f52"}
Jul 29 21:09:09.013293 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:08.869309    2580 scope.go:117] "RemoveContainer" containerID="e3063de80d0ecae28ba7c3bc2bcec164db61c130a2f71e626fb072427f098f52"
Jul 29 21:09:09.836454 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:09.836422    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7hv9z"
Jul 29 21:09:09.874640 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:09.874610    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg" event={"ID":"d135c4fb-0ebd-485a-8dbb-39369e431db7","Type":"ContainerStarted","Data":"2e0269ec442b4c69dd3eee449053565c23a559f96d9998c9efcff758ba163fbc"}
Jul 29 21:09:31.407048 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.406987    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-895466b4b-h9z7h" podUID="bd830671-514c-48d4-bd0e-b69694f7007d" containerName="console" containerID="cri-o://e27f25988bbced3d044f323c0f6c836a1a2593f7a600e7227e09cd7f80faca48" gracePeriod=15
Jul 29 21:09:31.683331 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.683261    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:09:31.757720 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.757685    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd830671-514c-48d4-bd0e-b69694f7007d-console-serving-cert\") pod \"bd830671-514c-48d4-bd0e-b69694f7007d\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") "
Jul 29 21:09:31.757931 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.757730    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhpxk\" (UniqueName: \"kubernetes.io/projected/bd830671-514c-48d4-bd0e-b69694f7007d-kube-api-access-hhpxk\") pod \"bd830671-514c-48d4-bd0e-b69694f7007d\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") "
Jul 29 21:09:31.757931 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.757759    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-service-ca\") pod \"bd830671-514c-48d4-bd0e-b69694f7007d\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") "
Jul 29 21:09:31.757931 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.757785    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-oauth-serving-cert\") pod \"bd830671-514c-48d4-bd0e-b69694f7007d\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") "
Jul 29 21:09:31.757931 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.757891    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd830671-514c-48d4-bd0e-b69694f7007d-console-oauth-config\") pod \"bd830671-514c-48d4-bd0e-b69694f7007d\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") "
Jul 29 21:09:31.757931 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.757923    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-console-config\") pod \"bd830671-514c-48d4-bd0e-b69694f7007d\" (UID: \"bd830671-514c-48d4-bd0e-b69694f7007d\") "
Jul 29 21:09:31.758300 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.758264    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-service-ca" (OuterVolumeSpecName: "service-ca") pod "bd830671-514c-48d4-bd0e-b69694f7007d" (UID: "bd830671-514c-48d4-bd0e-b69694f7007d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:09:31.758384 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.758298    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bd830671-514c-48d4-bd0e-b69694f7007d" (UID: "bd830671-514c-48d4-bd0e-b69694f7007d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:09:31.758384 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.758346    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-console-config" (OuterVolumeSpecName: "console-config") pod "bd830671-514c-48d4-bd0e-b69694f7007d" (UID: "bd830671-514c-48d4-bd0e-b69694f7007d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:09:31.760230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.760203    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd830671-514c-48d4-bd0e-b69694f7007d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bd830671-514c-48d4-bd0e-b69694f7007d" (UID: "bd830671-514c-48d4-bd0e-b69694f7007d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:09:31.760230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.760217    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd830671-514c-48d4-bd0e-b69694f7007d-kube-api-access-hhpxk" (OuterVolumeSpecName: "kube-api-access-hhpxk") pod "bd830671-514c-48d4-bd0e-b69694f7007d" (UID: "bd830671-514c-48d4-bd0e-b69694f7007d"). InnerVolumeSpecName "kube-api-access-hhpxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:09:31.760368 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.760293    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd830671-514c-48d4-bd0e-b69694f7007d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bd830671-514c-48d4-bd0e-b69694f7007d" (UID: "bd830671-514c-48d4-bd0e-b69694f7007d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:09:31.858997 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.858958    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd830671-514c-48d4-bd0e-b69694f7007d-console-oauth-config\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:09:31.858997 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.858989    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-console-config\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:09:31.858997 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.859000    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd830671-514c-48d4-bd0e-b69694f7007d-console-serving-cert\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:09:31.859229 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.859008    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhpxk\" (UniqueName: \"kubernetes.io/projected/bd830671-514c-48d4-bd0e-b69694f7007d-kube-api-access-hhpxk\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:09:31.859229 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.859018    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-service-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:09:31.859229 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.859027    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd830671-514c-48d4-bd0e-b69694f7007d-oauth-serving-cert\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:09:31.963533 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.963439    2580 generic.go:356] "Generic (PLEG): container finished" podID="bd830671-514c-48d4-bd0e-b69694f7007d" containerID="e27f25988bbced3d044f323c0f6c836a1a2593f7a600e7227e09cd7f80faca48" exitCode=0
Jul 29 21:09:31.963533 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.963489    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-895466b4b-h9z7h"
Jul 29 21:09:31.963533 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.963514    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-895466b4b-h9z7h" event={"ID":"bd830671-514c-48d4-bd0e-b69694f7007d","Type":"ContainerDied","Data":"e27f25988bbced3d044f323c0f6c836a1a2593f7a600e7227e09cd7f80faca48"}
Jul 29 21:09:31.963777 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.963564    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-895466b4b-h9z7h" event={"ID":"bd830671-514c-48d4-bd0e-b69694f7007d","Type":"ContainerDied","Data":"460206b49fe1d63424f9217281ec8f1e4221b8d79bae40bfa442f93323bb0f87"}
Jul 29 21:09:31.963777 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.963590    2580 scope.go:117] "RemoveContainer" containerID="e27f25988bbced3d044f323c0f6c836a1a2593f7a600e7227e09cd7f80faca48"
Jul 29 21:09:31.972136 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.972119    2580 scope.go:117] "RemoveContainer" containerID="e27f25988bbced3d044f323c0f6c836a1a2593f7a600e7227e09cd7f80faca48"
Jul 29 21:09:31.972380 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:09:31.972358    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e27f25988bbced3d044f323c0f6c836a1a2593f7a600e7227e09cd7f80faca48\": container with ID starting with e27f25988bbced3d044f323c0f6c836a1a2593f7a600e7227e09cd7f80faca48 not found: ID does not exist" containerID="e27f25988bbced3d044f323c0f6c836a1a2593f7a600e7227e09cd7f80faca48"
Jul 29 21:09:31.972440 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.972387    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e27f25988bbced3d044f323c0f6c836a1a2593f7a600e7227e09cd7f80faca48"} err="failed to get container status \"e27f25988bbced3d044f323c0f6c836a1a2593f7a600e7227e09cd7f80faca48\": rpc error: code = NotFound desc = could not find container \"e27f25988bbced3d044f323c0f6c836a1a2593f7a600e7227e09cd7f80faca48\": container with ID starting with e27f25988bbced3d044f323c0f6c836a1a2593f7a600e7227e09cd7f80faca48 not found: ID does not exist"
Jul 29 21:09:31.986053 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.986025    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-895466b4b-h9z7h"]
Jul 29 21:09:31.990805 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:31.990782    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-895466b4b-h9z7h"]
Jul 29 21:09:32.714790 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:32.714755    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd830671-514c-48d4-bd0e-b69694f7007d" path="/var/lib/kubelet/pods/bd830671-514c-48d4-bd0e-b69694f7007d/volumes"
Jul 29 21:09:41.149008 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.148965    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d8699c6f8-rjpvz"]
Jul 29 21:09:41.149599 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.149359    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd830671-514c-48d4-bd0e-b69694f7007d" containerName="console"
Jul 29 21:09:41.149599 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.149380    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd830671-514c-48d4-bd0e-b69694f7007d" containerName="console"
Jul 29 21:09:41.149599 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.149466    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd830671-514c-48d4-bd0e-b69694f7007d" containerName="console"
Jul 29 21:09:41.152630 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.152608    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.156737 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.156712    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-64d8z\""
Jul 29 21:09:41.156900 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.156730    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 21:09:41.156900 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.156712    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 21:09:41.156900 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.156787    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 21:09:41.157073 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.157003    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 21:09:41.157166 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.157147    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 21:09:41.162610 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.162561    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 21:09:41.164440 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.164409    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d8699c6f8-rjpvz"]
Jul 29 21:09:41.242018 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.241986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-service-ca\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.242018 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.242021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-trusted-ca-bundle\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.242213 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.242051    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-console-config\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.242213 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.242133    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4hpr\" (UniqueName: \"kubernetes.io/projected/2cb62361-4aa3-4b5f-816c-468ee8279104-kube-api-access-m4hpr\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.242213 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.242170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2cb62361-4aa3-4b5f-816c-468ee8279104-console-oauth-config\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.242213 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.242190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2cb62361-4aa3-4b5f-816c-468ee8279104-console-serving-cert\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.242339 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.242226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-oauth-serving-cert\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.342676 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.342634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-oauth-serving-cert\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.342676 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.342681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-service-ca\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.342979 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.342698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-trusted-ca-bundle\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.342979 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.342746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-console-config\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.342979 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.342789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4hpr\" (UniqueName: \"kubernetes.io/projected/2cb62361-4aa3-4b5f-816c-468ee8279104-kube-api-access-m4hpr\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.342979 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.342823    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2cb62361-4aa3-4b5f-816c-468ee8279104-console-oauth-config\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.342979 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.342875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2cb62361-4aa3-4b5f-816c-468ee8279104-console-serving-cert\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.343532 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.343502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-oauth-serving-cert\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.343655 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.343536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-service-ca\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.343655 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.343502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-console-config\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.343901 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.343884    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-trusted-ca-bundle\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.345326 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.345298    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2cb62361-4aa3-4b5f-816c-468ee8279104-console-oauth-config\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.345440 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.345423    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2cb62361-4aa3-4b5f-816c-468ee8279104-console-serving-cert\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.351555 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.351534    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4hpr\" (UniqueName: \"kubernetes.io/projected/2cb62361-4aa3-4b5f-816c-468ee8279104-kube-api-access-m4hpr\") pod \"console-5d8699c6f8-rjpvz\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") " pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.465492 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.465412    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:41.595424 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:41.595389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d8699c6f8-rjpvz"]
Jul 29 21:09:41.598194 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:09:41.598162    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb62361_4aa3_4b5f_816c_468ee8279104.slice/crio-c39e70c6a95d093c933535ac8a17a275bc1627c6574abfecc14f46530a5780ef WatchSource:0}: Error finding container c39e70c6a95d093c933535ac8a17a275bc1627c6574abfecc14f46530a5780ef: Status 404 returned error can't find the container with id c39e70c6a95d093c933535ac8a17a275bc1627c6574abfecc14f46530a5780ef
Jul 29 21:09:42.008503 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:42.008466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8699c6f8-rjpvz" event={"ID":"2cb62361-4aa3-4b5f-816c-468ee8279104","Type":"ContainerStarted","Data":"7d9b7fcc74892fc6496ad91cd2aec47952d2dc0699e167504a688bc795751364"}
Jul 29 21:09:42.008707 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:42.008518    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8699c6f8-rjpvz" event={"ID":"2cb62361-4aa3-4b5f-816c-468ee8279104","Type":"ContainerStarted","Data":"c39e70c6a95d093c933535ac8a17a275bc1627c6574abfecc14f46530a5780ef"}
Jul 29 21:09:42.027012 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:42.026946    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d8699c6f8-rjpvz" podStartSLOduration=1.026929913 podStartE2EDuration="1.026929913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:09:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:09:42.026508064 +0000 UTC m=+333.907782201" watchObservedRunningTime="2026-07-29 21:09:42.026929913 +0000 UTC m=+333.908204051"
Jul 29 21:09:51.465998 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:51.465954    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:51.466472 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:51.466016    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:51.470941 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:51.470918    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:09:52.054443 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:09:52.054412    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:11:14.893651 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:14.893613    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s"]
Jul 29 21:11:14.896098 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:14.896082    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s"
Jul 29 21:11:14.899058 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:14.899036    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 29 21:11:14.899195 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:14.899072    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 29 21:11:14.899195 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:14.899100    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-qql4h\""
Jul 29 21:11:14.909471 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:14.909446    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s"]
Jul 29 21:11:15.043537 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:15.043501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x55nf\" (UniqueName: \"kubernetes.io/projected/537ca494-e527-4a67-a741-df5de4ac4114-kube-api-access-x55nf\") pod \"cert-manager-operator-controller-manager-55c444bf86-6pk5s\" (UID: \"537ca494-e527-4a67-a741-df5de4ac4114\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s"
Jul 29 21:11:15.043719 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:15.043557    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/537ca494-e527-4a67-a741-df5de4ac4114-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6pk5s\" (UID: \"537ca494-e527-4a67-a741-df5de4ac4114\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s"
Jul 29 21:11:15.144138 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:15.144055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/537ca494-e527-4a67-a741-df5de4ac4114-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6pk5s\" (UID: \"537ca494-e527-4a67-a741-df5de4ac4114\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s"
Jul 29 21:11:15.144138 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:15.144129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x55nf\" (UniqueName: \"kubernetes.io/projected/537ca494-e527-4a67-a741-df5de4ac4114-kube-api-access-x55nf\") pod \"cert-manager-operator-controller-manager-55c444bf86-6pk5s\" (UID: \"537ca494-e527-4a67-a741-df5de4ac4114\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s"
Jul 29 21:11:15.144457 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:15.144437    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/537ca494-e527-4a67-a741-df5de4ac4114-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6pk5s\" (UID: \"537ca494-e527-4a67-a741-df5de4ac4114\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s"
Jul 29 21:11:15.153674 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:15.153642    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x55nf\" (UniqueName: \"kubernetes.io/projected/537ca494-e527-4a67-a741-df5de4ac4114-kube-api-access-x55nf\") pod \"cert-manager-operator-controller-manager-55c444bf86-6pk5s\" (UID: \"537ca494-e527-4a67-a741-df5de4ac4114\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s"
Jul 29 21:11:15.205233 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:15.205197    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s"
Jul 29 21:11:15.340012 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:15.339985    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s"]
Jul 29 21:11:15.342352 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:11:15.342326    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod537ca494_e527_4a67_a741_df5de4ac4114.slice/crio-c3642c600dc6065a6845c920fb8e68509f426eede6ee0d70e822d46ffe4ff9e0 WatchSource:0}: Error finding container c3642c600dc6065a6845c920fb8e68509f426eede6ee0d70e822d46ffe4ff9e0: Status 404 returned error can't find the container with id c3642c600dc6065a6845c920fb8e68509f426eede6ee0d70e822d46ffe4ff9e0
Jul 29 21:11:15.345054 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:15.345035    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:11:15.370303 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:15.370277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s" event={"ID":"537ca494-e527-4a67-a741-df5de4ac4114","Type":"ContainerStarted","Data":"c3642c600dc6065a6845c920fb8e68509f426eede6ee0d70e822d46ffe4ff9e0"}
Jul 29 21:11:18.385910 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:18.385874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s" event={"ID":"537ca494-e527-4a67-a741-df5de4ac4114","Type":"ContainerStarted","Data":"7abd312cba3b8e3bb8a8bfbb2be3556a17b3e3727dcc6f174d1ae17cb8098b29"}
Jul 29 21:11:18.408632 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:18.408580    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6pk5s" podStartSLOduration=2.00799489 podStartE2EDuration="4.408563706s" totalImagesPullingTime="2.400568816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:11:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:11:18.407656632 +0000 UTC m=+430.288930770" watchObservedRunningTime="2026-07-29 21:11:18.408563706 +0000 UTC m=+430.289837844"
Jul 29 21:11:20.939891 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:20.939852    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9hj98"]
Jul 29 21:11:20.942443 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:20.942426    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hj98"
Jul 29 21:11:20.945243 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:20.945222    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 21:11:20.946462 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:20.946445    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-zgsd7\""
Jul 29 21:11:20.946580 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:20.946467    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 21:11:20.951326 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:20.951305    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9hj98"]
Jul 29 21:11:20.995950 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:20.995915    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f49aa4e8-439e-40ca-ad5b-d836a231e609-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9hj98\" (UID: \"f49aa4e8-439e-40ca-ad5b-d836a231e609\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hj98"
Jul 29 21:11:20.995950 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:20.995952    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msd8b\" (UniqueName: \"kubernetes.io/projected/f49aa4e8-439e-40ca-ad5b-d836a231e609-kube-api-access-msd8b\") pod \"cert-manager-cainjector-559cdd8c87-9hj98\" (UID: \"f49aa4e8-439e-40ca-ad5b-d836a231e609\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hj98"
Jul 29 21:11:21.096708 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:21.096668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f49aa4e8-439e-40ca-ad5b-d836a231e609-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9hj98\" (UID: \"f49aa4e8-439e-40ca-ad5b-d836a231e609\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hj98"
Jul 29 21:11:21.096708 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:21.096709    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msd8b\" (UniqueName: \"kubernetes.io/projected/f49aa4e8-439e-40ca-ad5b-d836a231e609-kube-api-access-msd8b\") pod \"cert-manager-cainjector-559cdd8c87-9hj98\" (UID: \"f49aa4e8-439e-40ca-ad5b-d836a231e609\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hj98"
Jul 29 21:11:21.105287 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:21.105257    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f49aa4e8-439e-40ca-ad5b-d836a231e609-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9hj98\" (UID: \"f49aa4e8-439e-40ca-ad5b-d836a231e609\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hj98"
Jul 29 21:11:21.105395 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:21.105313    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msd8b\" (UniqueName: \"kubernetes.io/projected/f49aa4e8-439e-40ca-ad5b-d836a231e609-kube-api-access-msd8b\") pod \"cert-manager-cainjector-559cdd8c87-9hj98\" (UID: \"f49aa4e8-439e-40ca-ad5b-d836a231e609\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hj98"
Jul 29 21:11:21.252473 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:21.252391    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hj98"
Jul 29 21:11:21.381419 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:21.381394    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9hj98"]
Jul 29 21:11:21.383301 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:11:21.383273    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf49aa4e8_439e_40ca_ad5b_d836a231e609.slice/crio-5f53d300c18d0fdc5f7467c23f2ee11f128711728aecb18f38c9135249a1fb16 WatchSource:0}: Error finding container 5f53d300c18d0fdc5f7467c23f2ee11f128711728aecb18f38c9135249a1fb16: Status 404 returned error can't find the container with id 5f53d300c18d0fdc5f7467c23f2ee11f128711728aecb18f38c9135249a1fb16
Jul 29 21:11:21.399827 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:21.399797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hj98" event={"ID":"f49aa4e8-439e-40ca-ad5b-d836a231e609","Type":"ContainerStarted","Data":"5f53d300c18d0fdc5f7467c23f2ee11f128711728aecb18f38c9135249a1fb16"}
Jul 29 21:11:24.416660 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:24.416622    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hj98" event={"ID":"f49aa4e8-439e-40ca-ad5b-d836a231e609","Type":"ContainerStarted","Data":"04da3b750ebfbfbccecb701097604fd0f604816ef388df1507435f1cf348240f"}
Jul 29 21:11:24.438587 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:24.438494    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9hj98" podStartSLOduration=1.880008363 podStartE2EDuration="4.438476024s" totalImagesPullingTime="2.558467661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:11:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:11:24.438249641 +0000 UTC m=+436.319523779" watchObservedRunningTime="2026-07-29 21:11:24.438476024 +0000 UTC m=+436.319750162"
Jul 29 21:11:25.101587 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:25.101550    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-prrhx"]
Jul 29 21:11:25.104060 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:25.104044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-prrhx"
Jul 29 21:11:25.110611 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:25.110584    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-rwnkf\""
Jul 29 21:11:25.118217 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:25.118193    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-prrhx"]
Jul 29 21:11:25.123965 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:25.123940    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p5gt\" (UniqueName: \"kubernetes.io/projected/bc65db94-cbee-45c0-889e-19a5240880d0-kube-api-access-4p5gt\") pod \"cert-manager-webhook-6486ffd997-prrhx\" (UID: \"bc65db94-cbee-45c0-889e-19a5240880d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-prrhx"
Jul 29 21:11:25.124071 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:25.124004    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc65db94-cbee-45c0-889e-19a5240880d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-prrhx\" (UID: \"bc65db94-cbee-45c0-889e-19a5240880d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-prrhx"
Jul 29 21:11:25.224524 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:25.224485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc65db94-cbee-45c0-889e-19a5240880d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-prrhx\" (UID: \"bc65db94-cbee-45c0-889e-19a5240880d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-prrhx"
Jul 29 21:11:25.224685 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:25.224556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p5gt\" (UniqueName: \"kubernetes.io/projected/bc65db94-cbee-45c0-889e-19a5240880d0-kube-api-access-4p5gt\") pod \"cert-manager-webhook-6486ffd997-prrhx\" (UID: \"bc65db94-cbee-45c0-889e-19a5240880d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-prrhx"
Jul 29 21:11:25.233284 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:25.233254    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc65db94-cbee-45c0-889e-19a5240880d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-prrhx\" (UID: \"bc65db94-cbee-45c0-889e-19a5240880d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-prrhx"
Jul 29 21:11:25.233399 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:25.233310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p5gt\" (UniqueName: \"kubernetes.io/projected/bc65db94-cbee-45c0-889e-19a5240880d0-kube-api-access-4p5gt\") pod \"cert-manager-webhook-6486ffd997-prrhx\" (UID: \"bc65db94-cbee-45c0-889e-19a5240880d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-prrhx"
Jul 29 21:11:25.413258 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:25.413224    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-prrhx"
Jul 29 21:11:25.539722 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:25.539699    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-prrhx"]
Jul 29 21:11:25.541606 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:11:25.541578    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc65db94_cbee_45c0_889e_19a5240880d0.slice/crio-7ed93945b91656e8e3622e2ef236d82082373f2fc6d9fd6b917b4c38c51b1cb2 WatchSource:0}: Error finding container 7ed93945b91656e8e3622e2ef236d82082373f2fc6d9fd6b917b4c38c51b1cb2: Status 404 returned error can't find the container with id 7ed93945b91656e8e3622e2ef236d82082373f2fc6d9fd6b917b4c38c51b1cb2
Jul 29 21:11:26.427204 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:26.427159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-prrhx" event={"ID":"bc65db94-cbee-45c0-889e-19a5240880d0","Type":"ContainerStarted","Data":"952312f4b1d159f3e5226664063e2b67a439b161f71b610cc2780e64a2905596"}
Jul 29 21:11:26.427204 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:26.427206    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-prrhx" event={"ID":"bc65db94-cbee-45c0-889e-19a5240880d0","Type":"ContainerStarted","Data":"7ed93945b91656e8e3622e2ef236d82082373f2fc6d9fd6b917b4c38c51b1cb2"}
Jul 29 21:11:26.427451 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:26.427223    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-prrhx"
Jul 29 21:11:26.444854 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:26.444783    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-prrhx" podStartSLOduration=1.444766233 podStartE2EDuration="1.444766233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:11:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:11:26.4436291 +0000 UTC m=+438.324903241" watchObservedRunningTime="2026-07-29 21:11:26.444766233 +0000 UTC m=+438.326040371"
Jul 29 21:11:32.434648 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:32.434564    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-prrhx"
Jul 29 21:11:56.999599 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:56.999560    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"]
Jul 29 21:11:57.004555 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.004532    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.008345 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.008311    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 21:11:57.008345 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.008333    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 21:11:57.008516 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.008333    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 21:11:57.008516 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.008333    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k9mx9\""
Jul 29 21:11:57.008659 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.008646    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 21:11:57.018279 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.018250    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"]
Jul 29 21:11:57.099167 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.099124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5eefeb19-409d-4122-a96a-11db885cbb13-manifests\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.099167 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.099169    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mc54\" (UniqueName: \"kubernetes.io/projected/5eefeb19-409d-4122-a96a-11db885cbb13-kube-api-access-8mc54\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.099437 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.099257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5eefeb19-409d-4122-a96a-11db885cbb13-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.099437 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.099309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5eefeb19-409d-4122-a96a-11db885cbb13-webhook-cert\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.099437 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.099346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5eefeb19-409d-4122-a96a-11db885cbb13-tmp\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.200022 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.199975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5eefeb19-409d-4122-a96a-11db885cbb13-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.200230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.200029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5eefeb19-409d-4122-a96a-11db885cbb13-webhook-cert\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.200230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.200058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5eefeb19-409d-4122-a96a-11db885cbb13-tmp\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.200347 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.200224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5eefeb19-409d-4122-a96a-11db885cbb13-manifests\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.200347 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.200266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mc54\" (UniqueName: \"kubernetes.io/projected/5eefeb19-409d-4122-a96a-11db885cbb13-kube-api-access-8mc54\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.200589 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.200560    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5eefeb19-409d-4122-a96a-11db885cbb13-manifests\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.202500 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.202469    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5eefeb19-409d-4122-a96a-11db885cbb13-tmp\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.202602 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.202585    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5eefeb19-409d-4122-a96a-11db885cbb13-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.202693 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.202676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5eefeb19-409d-4122-a96a-11db885cbb13-webhook-cert\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.208371 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.208346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mc54\" (UniqueName: \"kubernetes.io/projected/5eefeb19-409d-4122-a96a-11db885cbb13-kube-api-access-8mc54\") pod \"opendatahub-operator-controller-manager-8446ffcd9c-zc97t\" (UID: \"5eefeb19-409d-4122-a96a-11db885cbb13\") " pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.315805 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.315701    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:11:57.456790 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.456768    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"]
Jul 29 21:11:57.459452 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:11:57.459421    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eefeb19_409d_4122_a96a_11db885cbb13.slice/crio-a161500e7ab40bf6197e97b700461881fbf838e7809f3fabfc7ba816c029e47e WatchSource:0}: Error finding container a161500e7ab40bf6197e97b700461881fbf838e7809f3fabfc7ba816c029e47e: Status 404 returned error can't find the container with id a161500e7ab40bf6197e97b700461881fbf838e7809f3fabfc7ba816c029e47e
Jul 29 21:11:57.566851 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:11:57.566747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t" event={"ID":"5eefeb19-409d-4122-a96a-11db885cbb13","Type":"ContainerStarted","Data":"a161500e7ab40bf6197e97b700461881fbf838e7809f3fabfc7ba816c029e47e"}
Jul 29 21:12:00.585097 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:00.585060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t" event={"ID":"5eefeb19-409d-4122-a96a-11db885cbb13","Type":"ContainerStarted","Data":"24eb5216eaf09671a3fddd1e1463668a1c50a79732a82247103cd21fa990a792"}
Jul 29 21:12:02.595264 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:02.595230    2580 generic.go:356] "Generic (PLEG): container finished" podID="5eefeb19-409d-4122-a96a-11db885cbb13" containerID="24eb5216eaf09671a3fddd1e1463668a1c50a79732a82247103cd21fa990a792" exitCode=0
Jul 29 21:12:02.595769 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:02.595310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t" event={"ID":"5eefeb19-409d-4122-a96a-11db885cbb13","Type":"ContainerDied","Data":"24eb5216eaf09671a3fddd1e1463668a1c50a79732a82247103cd21fa990a792"}
Jul 29 21:12:03.600866 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:03.600812    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t" event={"ID":"5eefeb19-409d-4122-a96a-11db885cbb13","Type":"ContainerStarted","Data":"56772ff28811d17c277836b99ae8b49f19dec354d779d1ccbf06c302cb11cfc1"}
Jul 29 21:12:03.601257 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:03.601145    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:12:03.623731 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:03.623676    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t" podStartSLOduration=3.154070432 podStartE2EDuration="7.623661821s" totalImagesPullingTime="3.38292126s" totalInitContainerRuntime="1.086670129s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:11:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:12:03.621136519 +0000 UTC m=+475.502410658" watchObservedRunningTime="2026-07-29 21:12:03.623661821 +0000 UTC m=+475.504936030"
Jul 29 21:12:14.607500 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:14.607467    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8446ffcd9c-zc97t"
Jul 29 21:12:16.879065 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:16.879028    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"]
Jul 29 21:12:16.887744 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:16.887719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"
Jul 29 21:12:16.893189 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:16.893148    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 21:12:16.893189 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:16.893155    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 21:12:16.893786 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:16.893762    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-kczcl\""
Jul 29 21:12:16.896162 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:16.896140    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"]
Jul 29 21:12:16.983628 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:16.983588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a389da2-ad1f-442a-aa1f-5727e3baa007-tmp\") pod \"kube-auth-proxy-75d94d9cdd-2dbzv\" (UID: \"9a389da2-ad1f-442a-aa1f-5727e3baa007\") " pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"
Jul 29 21:12:16.983813 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:16.983652    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a389da2-ad1f-442a-aa1f-5727e3baa007-tls-certs\") pod \"kube-auth-proxy-75d94d9cdd-2dbzv\" (UID: \"9a389da2-ad1f-442a-aa1f-5727e3baa007\") " pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"
Jul 29 21:12:16.983813 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:16.983684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7l8g\" (UniqueName: \"kubernetes.io/projected/9a389da2-ad1f-442a-aa1f-5727e3baa007-kube-api-access-k7l8g\") pod \"kube-auth-proxy-75d94d9cdd-2dbzv\" (UID: \"9a389da2-ad1f-442a-aa1f-5727e3baa007\") " pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"
Jul 29 21:12:17.084700 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:17.084668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a389da2-ad1f-442a-aa1f-5727e3baa007-tls-certs\") pod \"kube-auth-proxy-75d94d9cdd-2dbzv\" (UID: \"9a389da2-ad1f-442a-aa1f-5727e3baa007\") " pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"
Jul 29 21:12:17.084700 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:17.084706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7l8g\" (UniqueName: \"kubernetes.io/projected/9a389da2-ad1f-442a-aa1f-5727e3baa007-kube-api-access-k7l8g\") pod \"kube-auth-proxy-75d94d9cdd-2dbzv\" (UID: \"9a389da2-ad1f-442a-aa1f-5727e3baa007\") " pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"
Jul 29 21:12:17.084954 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:17.084761    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a389da2-ad1f-442a-aa1f-5727e3baa007-tmp\") pod \"kube-auth-proxy-75d94d9cdd-2dbzv\" (UID: \"9a389da2-ad1f-442a-aa1f-5727e3baa007\") " pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"
Jul 29 21:12:17.084954 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:12:17.084823    2580 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 29 21:12:17.084954 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:12:17.084931    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a389da2-ad1f-442a-aa1f-5727e3baa007-tls-certs podName:9a389da2-ad1f-442a-aa1f-5727e3baa007 nodeName:}" failed. No retries permitted until 2026-07-29 21:12:17.584913514 +0000 UTC m=+489.466187630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9a389da2-ad1f-442a-aa1f-5727e3baa007-tls-certs") pod "kube-auth-proxy-75d94d9cdd-2dbzv" (UID: "9a389da2-ad1f-442a-aa1f-5727e3baa007") : secret "kube-auth-proxy-tls" not found
Jul 29 21:12:17.087023 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:17.087003    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a389da2-ad1f-442a-aa1f-5727e3baa007-tmp\") pod \"kube-auth-proxy-75d94d9cdd-2dbzv\" (UID: \"9a389da2-ad1f-442a-aa1f-5727e3baa007\") " pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"
Jul 29 21:12:17.100482 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:17.100456    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7l8g\" (UniqueName: \"kubernetes.io/projected/9a389da2-ad1f-442a-aa1f-5727e3baa007-kube-api-access-k7l8g\") pod \"kube-auth-proxy-75d94d9cdd-2dbzv\" (UID: \"9a389da2-ad1f-442a-aa1f-5727e3baa007\") " pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"
Jul 29 21:12:17.590689 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:17.590651    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a389da2-ad1f-442a-aa1f-5727e3baa007-tls-certs\") pod \"kube-auth-proxy-75d94d9cdd-2dbzv\" (UID: \"9a389da2-ad1f-442a-aa1f-5727e3baa007\") " pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"
Jul 29 21:12:17.593151 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:17.593132    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a389da2-ad1f-442a-aa1f-5727e3baa007-tls-certs\") pod \"kube-auth-proxy-75d94d9cdd-2dbzv\" (UID: \"9a389da2-ad1f-442a-aa1f-5727e3baa007\") " pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"
Jul 29 21:12:17.798253 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:17.798223    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"
Jul 29 21:12:17.970671 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:17.970648    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv"]
Jul 29 21:12:17.972253 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:12:17.972222    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a389da2_ad1f_442a_aa1f_5727e3baa007.slice/crio-efe546e134eb9872b0ca5931b1f028e209eadb3fab557c9260c6cd16ac84a090 WatchSource:0}: Error finding container efe546e134eb9872b0ca5931b1f028e209eadb3fab557c9260c6cd16ac84a090: Status 404 returned error can't find the container with id efe546e134eb9872b0ca5931b1f028e209eadb3fab557c9260c6cd16ac84a090
Jul 29 21:12:18.666890 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:18.666832    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv" event={"ID":"9a389da2-ad1f-442a-aa1f-5727e3baa007","Type":"ContainerStarted","Data":"efe546e134eb9872b0ca5931b1f028e209eadb3fab557c9260c6cd16ac84a090"}
Jul 29 21:12:21.687919 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:21.687879    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv" event={"ID":"9a389da2-ad1f-442a-aa1f-5727e3baa007","Type":"ContainerStarted","Data":"6b0650e4d8cc8121e2a004ebc26ffb9ad36ccf09c46a98d210bdff030509c554"}
Jul 29 21:12:21.704098 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:21.704039    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-75d94d9cdd-2dbzv" podStartSLOduration=2.998791507 podStartE2EDuration="5.704022015s" totalImagesPullingTime="2.705230508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:12:21.702469232 +0000 UTC m=+493.583743369" watchObservedRunningTime="2026-07-29 21:12:21.704022015 +0000 UTC m=+493.585296194"
Jul 29 21:12:25.227785 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.227754    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"]
Jul 29 21:12:25.231099 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.231084    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.234230 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.234204    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 29 21:12:25.234379 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.234235    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 29 21:12:25.234379 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.234249    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jp8s2\""
Jul 29 21:12:25.234379 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.234204    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 21:12:25.234379 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.234211    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 29 21:12:25.242280 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.242257    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"]
Jul 29 21:12:25.355820 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.355786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4r48\" (UniqueName: \"kubernetes.io/projected/2598bc39-4fdd-4108-a10e-3b79e10fc161-kube-api-access-c4r48\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.356014 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.355859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2598bc39-4fdd-4108-a10e-3b79e10fc161-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.356014 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.355919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2598bc39-4fdd-4108-a10e-3b79e10fc161-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.356014 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.355990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2598bc39-4fdd-4108-a10e-3b79e10fc161-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.356121 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.356042    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2598bc39-4fdd-4108-a10e-3b79e10fc161-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.356121 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.356076    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2598bc39-4fdd-4108-a10e-3b79e10fc161-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.356121 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.356098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2598bc39-4fdd-4108-a10e-3b79e10fc161-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.456649 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.456614    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2598bc39-4fdd-4108-a10e-3b79e10fc161-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.456876 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.456666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2598bc39-4fdd-4108-a10e-3b79e10fc161-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.456876 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.456703    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2598bc39-4fdd-4108-a10e-3b79e10fc161-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.456876 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.456737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4r48\" (UniqueName: \"kubernetes.io/projected/2598bc39-4fdd-4108-a10e-3b79e10fc161-kube-api-access-c4r48\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.456876 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.456777    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2598bc39-4fdd-4108-a10e-3b79e10fc161-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.456876 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.456803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2598bc39-4fdd-4108-a10e-3b79e10fc161-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.456876 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.456831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2598bc39-4fdd-4108-a10e-3b79e10fc161-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.457481 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.457453    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2598bc39-4fdd-4108-a10e-3b79e10fc161-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.459348 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.459319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2598bc39-4fdd-4108-a10e-3b79e10fc161-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.459463 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.459358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2598bc39-4fdd-4108-a10e-3b79e10fc161-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.459536 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.459497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2598bc39-4fdd-4108-a10e-3b79e10fc161-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.459625 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.459607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2598bc39-4fdd-4108-a10e-3b79e10fc161-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.465513 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.465489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2598bc39-4fdd-4108-a10e-3b79e10fc161-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.466036 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.466017    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4r48\" (UniqueName: \"kubernetes.io/projected/2598bc39-4fdd-4108-a10e-3b79e10fc161-kube-api-access-c4r48\") pod \"istiod-openshift-gateway-5677ccb49b-hgj5f\" (UID: \"2598bc39-4fdd-4108-a10e-3b79e10fc161\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.541201 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.541111    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:25.674222 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.674199    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"]
Jul 29 21:12:25.676637 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:12:25.676609    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2598bc39_4fdd_4108_a10e_3b79e10fc161.slice/crio-546fd26fe708c4e3745b7f75ffa7f29928d5fa67a06e6b438af5daa31ccc8fb5 WatchSource:0}: Error finding container 546fd26fe708c4e3745b7f75ffa7f29928d5fa67a06e6b438af5daa31ccc8fb5: Status 404 returned error can't find the container with id 546fd26fe708c4e3745b7f75ffa7f29928d5fa67a06e6b438af5daa31ccc8fb5
Jul 29 21:12:25.705784 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:25.705751    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f" event={"ID":"2598bc39-4fdd-4108-a10e-3b79e10fc161","Type":"ContainerStarted","Data":"546fd26fe708c4e3745b7f75ffa7f29928d5fa67a06e6b438af5daa31ccc8fb5"}
Jul 29 21:12:28.150905 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.150857    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 29 21:12:28.151188 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.150940    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 29 21:12:28.204783 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.204760    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"]
Jul 29 21:12:28.210876 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.210857    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.215002 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.214978    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 29 21:12:28.215113 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.215056    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 29 21:12:28.215113 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.215081    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 29 21:12:28.215234 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.215083    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 21:12:28.215234 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.215006    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 21:12:28.215234 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.215003    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9lzxn\""
Jul 29 21:12:28.220273 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.220251    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"]
Jul 29 21:12:28.283728 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.283690    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8xr2\" (UniqueName: \"kubernetes.io/projected/8ee44a3c-344b-48b8-b9de-a59be3c58684-kube-api-access-z8xr2\") pod \"lws-controller-manager-d97fdbb4b-glrwk\" (UID: \"8ee44a3c-344b-48b8-b9de-a59be3c58684\") " pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.283928 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.283821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ee44a3c-344b-48b8-b9de-a59be3c58684-cert\") pod \"lws-controller-manager-d97fdbb4b-glrwk\" (UID: \"8ee44a3c-344b-48b8-b9de-a59be3c58684\") " pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.283928 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.283891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8ee44a3c-344b-48b8-b9de-a59be3c58684-manager-config\") pod \"lws-controller-manager-d97fdbb4b-glrwk\" (UID: \"8ee44a3c-344b-48b8-b9de-a59be3c58684\") " pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.284093 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.283954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ee44a3c-344b-48b8-b9de-a59be3c58684-metrics-cert\") pod \"lws-controller-manager-d97fdbb4b-glrwk\" (UID: \"8ee44a3c-344b-48b8-b9de-a59be3c58684\") " pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.384760 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.384721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ee44a3c-344b-48b8-b9de-a59be3c58684-cert\") pod \"lws-controller-manager-d97fdbb4b-glrwk\" (UID: \"8ee44a3c-344b-48b8-b9de-a59be3c58684\") " pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.384760 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.384766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8ee44a3c-344b-48b8-b9de-a59be3c58684-manager-config\") pod \"lws-controller-manager-d97fdbb4b-glrwk\" (UID: \"8ee44a3c-344b-48b8-b9de-a59be3c58684\") " pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.385064 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.384791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ee44a3c-344b-48b8-b9de-a59be3c58684-metrics-cert\") pod \"lws-controller-manager-d97fdbb4b-glrwk\" (UID: \"8ee44a3c-344b-48b8-b9de-a59be3c58684\") " pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.385064 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.384902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8xr2\" (UniqueName: \"kubernetes.io/projected/8ee44a3c-344b-48b8-b9de-a59be3c58684-kube-api-access-z8xr2\") pod \"lws-controller-manager-d97fdbb4b-glrwk\" (UID: \"8ee44a3c-344b-48b8-b9de-a59be3c58684\") " pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.385760 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.385703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8ee44a3c-344b-48b8-b9de-a59be3c58684-manager-config\") pod \"lws-controller-manager-d97fdbb4b-glrwk\" (UID: \"8ee44a3c-344b-48b8-b9de-a59be3c58684\") " pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.387858 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.387820    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ee44a3c-344b-48b8-b9de-a59be3c58684-cert\") pod \"lws-controller-manager-d97fdbb4b-glrwk\" (UID: \"8ee44a3c-344b-48b8-b9de-a59be3c58684\") " pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.387858 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.387832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ee44a3c-344b-48b8-b9de-a59be3c58684-metrics-cert\") pod \"lws-controller-manager-d97fdbb4b-glrwk\" (UID: \"8ee44a3c-344b-48b8-b9de-a59be3c58684\") " pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.400229 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.400172    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8xr2\" (UniqueName: \"kubernetes.io/projected/8ee44a3c-344b-48b8-b9de-a59be3c58684-kube-api-access-z8xr2\") pod \"lws-controller-manager-d97fdbb4b-glrwk\" (UID: \"8ee44a3c-344b-48b8-b9de-a59be3c58684\") " pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.521275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.521235    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:28.689719 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.689645    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"]
Jul 29 21:12:28.693528 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:12:28.693495    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee44a3c_344b_48b8_b9de_a59be3c58684.slice/crio-b8826bf7df65f9f01edce1d3bc06f9b481aa0e1817a2c5e87b6fce0966cf1f5c WatchSource:0}: Error finding container b8826bf7df65f9f01edce1d3bc06f9b481aa0e1817a2c5e87b6fce0966cf1f5c: Status 404 returned error can't find the container with id b8826bf7df65f9f01edce1d3bc06f9b481aa0e1817a2c5e87b6fce0966cf1f5c
Jul 29 21:12:28.724112 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.724056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk" event={"ID":"8ee44a3c-344b-48b8-b9de-a59be3c58684","Type":"ContainerStarted","Data":"b8826bf7df65f9f01edce1d3bc06f9b481aa0e1817a2c5e87b6fce0966cf1f5c"}
Jul 29 21:12:28.726088 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.726062    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f" event={"ID":"2598bc39-4fdd-4108-a10e-3b79e10fc161","Type":"ContainerStarted","Data":"caf75e898127c8ef40e7ece2b583b9f5f4929305f1767249e6b37ccecfa72192"}
Jul 29 21:12:28.726225 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.726175    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:28.749986 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:28.749935    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f" podStartSLOduration=1.278503958 podStartE2EDuration="3.749919284s" totalImagesPullingTime="2.471415326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:12:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:12:28.748085021 +0000 UTC m=+500.629359161" watchObservedRunningTime="2026-07-29 21:12:28.749919284 +0000 UTC m=+500.631193421"
Jul 29 21:12:29.735159 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:29.735126    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hgj5f"
Jul 29 21:12:31.743019 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:31.742975    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk" event={"ID":"8ee44a3c-344b-48b8-b9de-a59be3c58684","Type":"ContainerStarted","Data":"1703a6b856fda305a7f0e89f38dd575e1996310807937979b2ec3427c0f789fa"}
Jul 29 21:12:31.743477 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:31.743079    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:12:31.760943 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:31.760890    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk" podStartSLOduration=1.087567443 podStartE2EDuration="3.760868785s" totalImagesPullingTime="2.673301342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:12:31.758964438 +0000 UTC m=+503.640238595" watchObservedRunningTime="2026-07-29 21:12:31.760868785 +0000 UTC m=+503.642142926"
Jul 29 21:12:42.749219 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:12:42.749190    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d97fdbb4b-glrwk"
Jul 29 21:14:01.653667 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:01.653623    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d8699c6f8-rjpvz"]
Jul 29 21:14:08.527530 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:08.527491    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:14:08.528110 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:08.528089    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:14:08.532125 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:08.532105    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:14:08.532887 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:08.532870    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:14:26.675528 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.675460    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d8699c6f8-rjpvz" podUID="2cb62361-4aa3-4b5f-816c-468ee8279104" containerName="console" containerID="cri-o://7d9b7fcc74892fc6496ad91cd2aec47952d2dc0699e167504a688bc795751364" gracePeriod=15
Jul 29 21:14:26.924691 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.924667    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:14:26.979125 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.979040    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2cb62361-4aa3-4b5f-816c-468ee8279104-console-oauth-config\") pod \"2cb62361-4aa3-4b5f-816c-468ee8279104\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") "
Jul 29 21:14:26.979125 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.979078    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-console-config\") pod \"2cb62361-4aa3-4b5f-816c-468ee8279104\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") "
Jul 29 21:14:26.979125 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.979109    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-service-ca\") pod \"2cb62361-4aa3-4b5f-816c-468ee8279104\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") "
Jul 29 21:14:26.979390 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.979131    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4hpr\" (UniqueName: \"kubernetes.io/projected/2cb62361-4aa3-4b5f-816c-468ee8279104-kube-api-access-m4hpr\") pod \"2cb62361-4aa3-4b5f-816c-468ee8279104\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") "
Jul 29 21:14:26.979390 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.979176    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2cb62361-4aa3-4b5f-816c-468ee8279104-console-serving-cert\") pod \"2cb62361-4aa3-4b5f-816c-468ee8279104\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") "
Jul 29 21:14:26.979390 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.979193    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-trusted-ca-bundle\") pod \"2cb62361-4aa3-4b5f-816c-468ee8279104\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") "
Jul 29 21:14:26.979390 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.979214    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-oauth-serving-cert\") pod \"2cb62361-4aa3-4b5f-816c-468ee8279104\" (UID: \"2cb62361-4aa3-4b5f-816c-468ee8279104\") "
Jul 29 21:14:26.979605 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.979481    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-console-config" (OuterVolumeSpecName: "console-config") pod "2cb62361-4aa3-4b5f-816c-468ee8279104" (UID: "2cb62361-4aa3-4b5f-816c-468ee8279104"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:14:26.979659 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.979522    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-service-ca" (OuterVolumeSpecName: "service-ca") pod "2cb62361-4aa3-4b5f-816c-468ee8279104" (UID: "2cb62361-4aa3-4b5f-816c-468ee8279104"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:14:26.979659 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.979621    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2cb62361-4aa3-4b5f-816c-468ee8279104" (UID: "2cb62361-4aa3-4b5f-816c-468ee8279104"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:14:26.979659 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.979638    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2cb62361-4aa3-4b5f-816c-468ee8279104" (UID: "2cb62361-4aa3-4b5f-816c-468ee8279104"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:14:26.981523 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.981495    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cb62361-4aa3-4b5f-816c-468ee8279104-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2cb62361-4aa3-4b5f-816c-468ee8279104" (UID: "2cb62361-4aa3-4b5f-816c-468ee8279104"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:14:26.981600 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.981561    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cb62361-4aa3-4b5f-816c-468ee8279104-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2cb62361-4aa3-4b5f-816c-468ee8279104" (UID: "2cb62361-4aa3-4b5f-816c-468ee8279104"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:14:26.981600 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:26.981577    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cb62361-4aa3-4b5f-816c-468ee8279104-kube-api-access-m4hpr" (OuterVolumeSpecName: "kube-api-access-m4hpr") pod "2cb62361-4aa3-4b5f-816c-468ee8279104" (UID: "2cb62361-4aa3-4b5f-816c-468ee8279104"). InnerVolumeSpecName "kube-api-access-m4hpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:14:27.080189 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.080139    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-service-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:14:27.080189 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.080185    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4hpr\" (UniqueName: \"kubernetes.io/projected/2cb62361-4aa3-4b5f-816c-468ee8279104-kube-api-access-m4hpr\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:14:27.080189 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.080197    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2cb62361-4aa3-4b5f-816c-468ee8279104-console-serving-cert\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:14:27.080189 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.080206    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-trusted-ca-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:14:27.080457 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.080215    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-oauth-serving-cert\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:14:27.080457 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.080223    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2cb62361-4aa3-4b5f-816c-468ee8279104-console-oauth-config\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:14:27.080457 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.080231    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2cb62361-4aa3-4b5f-816c-468ee8279104-console-config\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:14:27.297697 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.297610    2580 generic.go:356] "Generic (PLEG): container finished" podID="2cb62361-4aa3-4b5f-816c-468ee8279104" containerID="7d9b7fcc74892fc6496ad91cd2aec47952d2dc0699e167504a688bc795751364" exitCode=0
Jul 29 21:14:27.297697 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.297657    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8699c6f8-rjpvz"
Jul 29 21:14:27.297951 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.297702    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8699c6f8-rjpvz" event={"ID":"2cb62361-4aa3-4b5f-816c-468ee8279104","Type":"ContainerDied","Data":"7d9b7fcc74892fc6496ad91cd2aec47952d2dc0699e167504a688bc795751364"}
Jul 29 21:14:27.297951 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.297756    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8699c6f8-rjpvz" event={"ID":"2cb62361-4aa3-4b5f-816c-468ee8279104","Type":"ContainerDied","Data":"c39e70c6a95d093c933535ac8a17a275bc1627c6574abfecc14f46530a5780ef"}
Jul 29 21:14:27.297951 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.297790    2580 scope.go:117] "RemoveContainer" containerID="7d9b7fcc74892fc6496ad91cd2aec47952d2dc0699e167504a688bc795751364"
Jul 29 21:14:27.306950 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.306933    2580 scope.go:117] "RemoveContainer" containerID="7d9b7fcc74892fc6496ad91cd2aec47952d2dc0699e167504a688bc795751364"
Jul 29 21:14:27.307211 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:14:27.307194    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d9b7fcc74892fc6496ad91cd2aec47952d2dc0699e167504a688bc795751364\": container with ID starting with 7d9b7fcc74892fc6496ad91cd2aec47952d2dc0699e167504a688bc795751364 not found: ID does not exist" containerID="7d9b7fcc74892fc6496ad91cd2aec47952d2dc0699e167504a688bc795751364"
Jul 29 21:14:27.307275 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.307221    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d9b7fcc74892fc6496ad91cd2aec47952d2dc0699e167504a688bc795751364"} err="failed to get container status \"7d9b7fcc74892fc6496ad91cd2aec47952d2dc0699e167504a688bc795751364\": rpc error: code = NotFound desc = could not find container \"7d9b7fcc74892fc6496ad91cd2aec47952d2dc0699e167504a688bc795751364\": container with ID starting with 7d9b7fcc74892fc6496ad91cd2aec47952d2dc0699e167504a688bc795751364 not found: ID does not exist"
Jul 29 21:14:27.319154 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.319129    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d8699c6f8-rjpvz"]
Jul 29 21:14:27.322878 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:27.322857    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d8699c6f8-rjpvz"]
Jul 29 21:14:28.714868 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:14:28.714822    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cb62361-4aa3-4b5f-816c-468ee8279104" path="/var/lib/kubelet/pods/2cb62361-4aa3-4b5f-816c-468ee8279104/volumes"
Jul 29 21:19:08.558952 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:19:08.558873    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:19:08.561040 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:19:08.561019    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:19:08.563658 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:19:08.563640    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:19:08.565290 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:19:08.565274    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:24:08.592829 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:24:08.592798    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:24:08.595917 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:24:08.595894    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:24:08.596797 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:24:08.596777    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:24:08.599768 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:24:08.599743    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:29:08.619883 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:29:08.619831    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:29:08.624144 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:29:08.624120    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:29:08.624327 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:29:08.624200    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:29:08.628551 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:29:08.628528    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:34:08.654524 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:34:08.654488    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:34:08.657327 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:34:08.657303    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:34:08.658431 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:34:08.658411    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:34:08.661362 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:34:08.661338    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:39:08.682212 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:39:08.682177    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:39:08.686601 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:39:08.686575    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:39:08.689265 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:39:08.689238    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:39:08.694479 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:39:08.694446    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:44:08.712401 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:08.712353    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:44:08.717098 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:08.717072    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:44:08.721075 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:08.721051    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:44:08.725222 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:08.725198    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:44:14.201111 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.201070    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"]
Jul 29 21:44:14.201579 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.201564    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb62361-4aa3-4b5f-816c-468ee8279104" containerName="console"
Jul 29 21:44:14.201634 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.201583    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb62361-4aa3-4b5f-816c-468ee8279104" containerName="console"
Jul 29 21:44:14.201685 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.201673    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cb62361-4aa3-4b5f-816c-468ee8279104" containerName="console"
Jul 29 21:44:14.205015 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.204996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.209394 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.209364    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-69c0f1\"/\"kube-root-ca.crt\""
Jul 29 21:44:14.209539 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.209403    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-69c0f1\"/\"test-model-69c0f1-kserve-self-signed-certs\""
Jul 29 21:44:14.209539 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.209404    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-69c0f1\"/\"default-dockercfg-5bzkt\""
Jul 29 21:44:14.209539 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.209363    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-69c0f1\"/\"openshift-service-ca.crt\""
Jul 29 21:44:14.213462 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.213424    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"]
Jul 29 21:44:14.296135 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.296096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-dshm\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.296321 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.296164    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-tmp-dir\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.296321 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.296185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/825252da-f591-4e4a-bfdf-07070ccbba37-tls-certs\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.296321 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.296203    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-model-cache\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.296321 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.296225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-home\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.296321 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.296240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz5xg\" (UniqueName: \"kubernetes.io/projected/825252da-f591-4e4a-bfdf-07070ccbba37-kube-api-access-bz5xg\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.397220 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.397182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-dshm\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.397402 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.397331    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-tmp-dir\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.397402 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.397354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/825252da-f591-4e4a-bfdf-07070ccbba37-tls-certs\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.397402 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.397373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-model-cache\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.397552 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.397401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-home\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.397552 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.397425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz5xg\" (UniqueName: \"kubernetes.io/projected/825252da-f591-4e4a-bfdf-07070ccbba37-kube-api-access-bz5xg\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.397774 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.397748    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-tmp-dir\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.397882 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.397790    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-model-cache\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.397882 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.397828    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-home\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.399964 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.399921    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-dshm\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.400358 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.400341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/825252da-f591-4e4a-bfdf-07070ccbba37-tls-certs\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.406643 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.406616    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz5xg\" (UniqueName: \"kubernetes.io/projected/825252da-f591-4e4a-bfdf-07070ccbba37-kube-api-access-bz5xg\") pod \"test-model-69c0f1-kserve-6f6794bb7f-lcwpn\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") " pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.515497 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.515394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:14.654884 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.654853    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"]
Jul 29 21:44:14.656915 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:44:14.656872    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod825252da_f591_4e4a_bfdf_07070ccbba37.slice/crio-736ced2a7c19174d8cd1c5b2bf6845d4785e446c7ba5ba138494c1662bffcd52 WatchSource:0}: Error finding container 736ced2a7c19174d8cd1c5b2bf6845d4785e446c7ba5ba138494c1662bffcd52: Status 404 returned error can't find the container with id 736ced2a7c19174d8cd1c5b2bf6845d4785e446c7ba5ba138494c1662bffcd52
Jul 29 21:44:14.658937 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:14.658914    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:44:15.509760 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:15.509695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn" event={"ID":"825252da-f591-4e4a-bfdf-07070ccbba37","Type":"ContainerStarted","Data":"736ced2a7c19174d8cd1c5b2bf6845d4785e446c7ba5ba138494c1662bffcd52"}
Jul 29 21:44:16.521760 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:16.521718    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn" event={"ID":"825252da-f591-4e4a-bfdf-07070ccbba37","Type":"ContainerStarted","Data":"2c1d0575c20be1b790595e67ba158aa210b3d2cbac608e8c81d0e2ee910ed476"}
Jul 29 21:44:16.540856 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:16.540792    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn" podStartSLOduration=1.579555606 podStartE2EDuration="2.540773191s" totalImagesPullingTime="961.217585ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:44:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:44:16.53874792 +0000 UTC m=+2408.420022059" watchObservedRunningTime="2026-07-29 21:44:16.540773191 +0000 UTC m=+2408.422047329"
Jul 29 21:44:24.515736 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:24.515684    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:24.516205 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:24.515955    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:24.528708 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:24.528676    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:44:24.569208 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:44:24.569175    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:45:26.113465 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:26.113422    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"]
Jul 29 21:45:41.133638 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.133593    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn" podUID="825252da-f591-4e4a-bfdf-07070ccbba37" containerName="main" containerID="cri-o://2c1d0575c20be1b790595e67ba158aa210b3d2cbac608e8c81d0e2ee910ed476" gracePeriod=45
Jul 29 21:45:41.379912 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.379885    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:45:41.468324 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.468212    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-home\") pod \"825252da-f591-4e4a-bfdf-07070ccbba37\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") "
Jul 29 21:45:41.468324 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.468271    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-tmp-dir\") pod \"825252da-f591-4e4a-bfdf-07070ccbba37\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") "
Jul 29 21:45:41.468324 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.468287    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-model-cache\") pod \"825252da-f591-4e4a-bfdf-07070ccbba37\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") "
Jul 29 21:45:41.468636 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.468354    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-dshm\") pod \"825252da-f591-4e4a-bfdf-07070ccbba37\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") "
Jul 29 21:45:41.468636 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.468397    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz5xg\" (UniqueName: \"kubernetes.io/projected/825252da-f591-4e4a-bfdf-07070ccbba37-kube-api-access-bz5xg\") pod \"825252da-f591-4e4a-bfdf-07070ccbba37\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") "
Jul 29 21:45:41.468636 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.468426    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/825252da-f591-4e4a-bfdf-07070ccbba37-tls-certs\") pod \"825252da-f591-4e4a-bfdf-07070ccbba37\" (UID: \"825252da-f591-4e4a-bfdf-07070ccbba37\") "
Jul 29 21:45:41.468636 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.468506    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "825252da-f591-4e4a-bfdf-07070ccbba37" (UID: "825252da-f591-4e4a-bfdf-07070ccbba37"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:45:41.468636 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.468537    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-home" (OuterVolumeSpecName: "home") pod "825252da-f591-4e4a-bfdf-07070ccbba37" (UID: "825252da-f591-4e4a-bfdf-07070ccbba37"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:45:41.468636 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.468552    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-model-cache" (OuterVolumeSpecName: "model-cache") pod "825252da-f591-4e4a-bfdf-07070ccbba37" (UID: "825252da-f591-4e4a-bfdf-07070ccbba37"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:45:41.468934 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.468731    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-home\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:45:41.468934 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.468748    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-tmp-dir\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:45:41.468934 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.468757    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-model-cache\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:45:41.470634 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.470600    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/825252da-f591-4e4a-bfdf-07070ccbba37-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "825252da-f591-4e4a-bfdf-07070ccbba37" (UID: "825252da-f591-4e4a-bfdf-07070ccbba37"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:45:41.470634 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.470610    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-dshm" (OuterVolumeSpecName: "dshm") pod "825252da-f591-4e4a-bfdf-07070ccbba37" (UID: "825252da-f591-4e4a-bfdf-07070ccbba37"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:45:41.470791 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.470722    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/825252da-f591-4e4a-bfdf-07070ccbba37-kube-api-access-bz5xg" (OuterVolumeSpecName: "kube-api-access-bz5xg") pod "825252da-f591-4e4a-bfdf-07070ccbba37" (UID: "825252da-f591-4e4a-bfdf-07070ccbba37"). InnerVolumeSpecName "kube-api-access-bz5xg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:45:41.569413 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.569376    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/825252da-f591-4e4a-bfdf-07070ccbba37-dshm\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:45:41.569413 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.569407    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz5xg\" (UniqueName: \"kubernetes.io/projected/825252da-f591-4e4a-bfdf-07070ccbba37-kube-api-access-bz5xg\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:45:41.569413 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.569419    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/825252da-f591-4e4a-bfdf-07070ccbba37-tls-certs\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:45:41.913134 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.913100    2580 generic.go:356] "Generic (PLEG): container finished" podID="825252da-f591-4e4a-bfdf-07070ccbba37" containerID="2c1d0575c20be1b790595e67ba158aa210b3d2cbac608e8c81d0e2ee910ed476" exitCode=0
Jul 29 21:45:41.913314 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.913139    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"
Jul 29 21:45:41.913314 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.913180    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn" event={"ID":"825252da-f591-4e4a-bfdf-07070ccbba37","Type":"ContainerDied","Data":"2c1d0575c20be1b790595e67ba158aa210b3d2cbac608e8c81d0e2ee910ed476"}
Jul 29 21:45:41.913314 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.913228    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn" event={"ID":"825252da-f591-4e4a-bfdf-07070ccbba37","Type":"ContainerDied","Data":"736ced2a7c19174d8cd1c5b2bf6845d4785e446c7ba5ba138494c1662bffcd52"}
Jul 29 21:45:41.913314 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.913250    2580 scope.go:117] "RemoveContainer" containerID="2c1d0575c20be1b790595e67ba158aa210b3d2cbac608e8c81d0e2ee910ed476"
Jul 29 21:45:41.922276 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.922261    2580 scope.go:117] "RemoveContainer" containerID="2c1d0575c20be1b790595e67ba158aa210b3d2cbac608e8c81d0e2ee910ed476"
Jul 29 21:45:41.922537 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:45:41.922518    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c1d0575c20be1b790595e67ba158aa210b3d2cbac608e8c81d0e2ee910ed476\": container with ID starting with 2c1d0575c20be1b790595e67ba158aa210b3d2cbac608e8c81d0e2ee910ed476 not found: ID does not exist" containerID="2c1d0575c20be1b790595e67ba158aa210b3d2cbac608e8c81d0e2ee910ed476"
Jul 29 21:45:41.922603 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.922548    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c1d0575c20be1b790595e67ba158aa210b3d2cbac608e8c81d0e2ee910ed476"} err="failed to get container status \"2c1d0575c20be1b790595e67ba158aa210b3d2cbac608e8c81d0e2ee910ed476\": rpc error: code = NotFound desc = could not find container \"2c1d0575c20be1b790595e67ba158aa210b3d2cbac608e8c81d0e2ee910ed476\": container with ID starting with 2c1d0575c20be1b790595e67ba158aa210b3d2cbac608e8c81d0e2ee910ed476 not found: ID does not exist"
Jul 29 21:45:41.934274 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.934247    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"]
Jul 29 21:45:41.938009 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:41.937981    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-69c0f1/test-model-69c0f1-kserve-6f6794bb7f-lcwpn"]
Jul 29 21:45:42.714745 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:45:42.714712    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="825252da-f591-4e4a-bfdf-07070ccbba37" path="/var/lib/kubelet/pods/825252da-f591-4e4a-bfdf-07070ccbba37/volumes"
Jul 29 21:48:52.090380 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.090344    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"]
Jul 29 21:48:52.090782 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.090642    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="825252da-f591-4e4a-bfdf-07070ccbba37" containerName="main"
Jul 29 21:48:52.090782 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.090653    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="825252da-f591-4e4a-bfdf-07070ccbba37" containerName="main"
Jul 29 21:48:52.090782 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.090737    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="825252da-f591-4e4a-bfdf-07070ccbba37" containerName="main"
Jul 29 21:48:52.093494 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.093475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.097930 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.097911    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fe32bc\"/\"openshift-service-ca.crt\""
Jul 29 21:48:52.097930 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.097921    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fe32bc\"/\"default-dockercfg-7gwbz\""
Jul 29 21:48:52.098096 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.097950    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fe32bc\"/\"auth-test-model-fe32bc-kserve-self-signed-certs\""
Jul 29 21:48:52.098096 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.098059    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fe32bc\"/\"kube-root-ca.crt\""
Jul 29 21:48:52.103368 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.103349    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"]
Jul 29 21:48:52.161403 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.161374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-home\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.161524 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.161408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6348eed9-db45-40f4-ad0d-4112261502be-tls-certs\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.161524 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.161439    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-dshm\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.161524 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.161504    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-tmp-dir\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.161660 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.161555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-model-cache\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.161660 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.161581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2rt6\" (UniqueName: \"kubernetes.io/projected/6348eed9-db45-40f4-ad0d-4112261502be-kube-api-access-b2rt6\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.262650 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.262624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-home\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.262766 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.262653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6348eed9-db45-40f4-ad0d-4112261502be-tls-certs\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.262766 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.262673    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-dshm\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.262766 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.262701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-tmp-dir\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.262766 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.262752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-model-cache\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.262972 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.262786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2rt6\" (UniqueName: \"kubernetes.io/projected/6348eed9-db45-40f4-ad0d-4112261502be-kube-api-access-b2rt6\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.263065 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.263046    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-home\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.263189 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.263170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-model-cache\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.263252 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.263203    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-tmp-dir\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.265113 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.265096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-dshm\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.265241 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.265226    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6348eed9-db45-40f4-ad0d-4112261502be-tls-certs\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.273171 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.273151    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2rt6\" (UniqueName: \"kubernetes.io/projected/6348eed9-db45-40f4-ad0d-4112261502be-kube-api-access-b2rt6\") pod \"auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") " pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.403089 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.403063    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:48:52.533134 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.533111    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"]
Jul 29 21:48:52.535122 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:48:52.535098    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6348eed9_db45_40f4_ad0d_4112261502be.slice/crio-2eb062bfceaedcc396611e5659c09cf2a473e27bbbb71e8ae460761039a82670 WatchSource:0}: Error finding container 2eb062bfceaedcc396611e5659c09cf2a473e27bbbb71e8ae460761039a82670: Status 404 returned error can't find the container with id 2eb062bfceaedcc396611e5659c09cf2a473e27bbbb71e8ae460761039a82670
Jul 29 21:48:52.746321 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.746233    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd" event={"ID":"6348eed9-db45-40f4-ad0d-4112261502be","Type":"ContainerStarted","Data":"8ec2def8e407f23e011f3bfebfcc18c13448674430cdb4320e3a2b05daeceb06"}
Jul 29 21:48:52.746321 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.746275    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd" event={"ID":"6348eed9-db45-40f4-ad0d-4112261502be","Type":"ContainerStarted","Data":"2eb062bfceaedcc396611e5659c09cf2a473e27bbbb71e8ae460761039a82670"}
Jul 29 21:48:52.765311 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:48:52.765260    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd" podStartSLOduration=0.765245732 podStartE2EDuration="765.245732ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:48:52.76309226 +0000 UTC m=+2684.644366397" watchObservedRunningTime="2026-07-29 21:48:52.765245732 +0000 UTC m=+2684.646519870"
Jul 29 21:49:02.403945 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:02.403863    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:49:02.403945 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:02.403915    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:49:02.416404 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:02.416380    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:49:02.799155 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:02.799066    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:49:08.739392 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:08.739363    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:49:08.743194 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:08.743175    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:49:08.748598 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:08.748579    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:49:08.752646 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:08.752630    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:49:21.001159 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.001123    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"]
Jul 29 21:49:21.005012 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.004989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.007972 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.007948    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-670ddc\"/\"openshift-service-ca.crt\""
Jul 29 21:49:21.009277 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.009260    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-670ddc\"/\"kube-root-ca.crt\""
Jul 29 21:49:21.009383 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.009282    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-670ddc\"/\"default-dockercfg-82w8b\""
Jul 29 21:49:21.009383 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.009294    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-670ddc\"/\"auth-test-model-670ddc-kserve-self-signed-certs\""
Jul 29 21:49:21.014395 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.014354    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"]
Jul 29 21:49:21.095443 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.095414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02d5d2f0-6588-4288-9635-8237a9391bb8-tls-certs\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.095591 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.095449    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-model-cache\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.095591 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.095477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-tmp-dir\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.095591 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.095574    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-dshm\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.095713 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.095605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-home\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.095713 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.095676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qfl8\" (UniqueName: \"kubernetes.io/projected/02d5d2f0-6588-4288-9635-8237a9391bb8-kube-api-access-6qfl8\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.196483 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.196452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qfl8\" (UniqueName: \"kubernetes.io/projected/02d5d2f0-6588-4288-9635-8237a9391bb8-kube-api-access-6qfl8\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.196617 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.196493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02d5d2f0-6588-4288-9635-8237a9391bb8-tls-certs\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.196617 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.196515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-model-cache\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.196617 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.196541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-tmp-dir\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.196617 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.196566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-dshm\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.196617 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.196586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-home\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.197074 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.197053    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-model-cache\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.197074 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.197068    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-tmp-dir\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.197223 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.197125    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-home\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.198957 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.198936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-dshm\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.199083 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.199061    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02d5d2f0-6588-4288-9635-8237a9391bb8-tls-certs\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.205693 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.205661    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qfl8\" (UniqueName: \"kubernetes.io/projected/02d5d2f0-6588-4288-9635-8237a9391bb8-kube-api-access-6qfl8\") pod \"auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") " pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.315799 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.315733    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:21.443662 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.443637    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"]
Jul 29 21:49:21.445537 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:49:21.445510    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02d5d2f0_6588_4288_9635_8237a9391bb8.slice/crio-faf1efebe7a4a0c432d6d746fbdbcf370395068e0f5b2458790a41a543f7b0fa WatchSource:0}: Error finding container faf1efebe7a4a0c432d6d746fbdbcf370395068e0f5b2458790a41a543f7b0fa: Status 404 returned error can't find the container with id faf1efebe7a4a0c432d6d746fbdbcf370395068e0f5b2458790a41a543f7b0fa
Jul 29 21:49:21.875004 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.874970    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl" event={"ID":"02d5d2f0-6588-4288-9635-8237a9391bb8","Type":"ContainerStarted","Data":"3251d745cc0807059c9ecc7c4fa1d5755d8e99a95c9102310f23a726a6e7c640"}
Jul 29 21:49:21.875004 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.875010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl" event={"ID":"02d5d2f0-6588-4288-9635-8237a9391bb8","Type":"ContainerStarted","Data":"faf1efebe7a4a0c432d6d746fbdbcf370395068e0f5b2458790a41a543f7b0fa"}
Jul 29 21:49:21.895021 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:21.894969    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl" podStartSLOduration=1.894954302 podStartE2EDuration="1.894954302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:49:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:49:21.893452571 +0000 UTC m=+2713.774726710" watchObservedRunningTime="2026-07-29 21:49:21.894954302 +0000 UTC m=+2713.776228440"
Jul 29 21:49:31.316103 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:31.316052    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:31.316586 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:31.316315    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:31.328582 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:31.328561    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:49:31.932581 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:49:31.932549    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:54:04.224444 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:04.224367    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"]
Jul 29 21:54:05.016418 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:05.016388    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"]
Jul 29 21:54:08.768193 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:08.768150    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:54:08.773122 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:08.773095    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:54:08.779210 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:08.779187    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:54:08.783218 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:08.783202    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:54:19.242652 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.242614    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd" podUID="6348eed9-db45-40f4-ad0d-4112261502be" containerName="main" containerID="cri-o://8ec2def8e407f23e011f3bfebfcc18c13448674430cdb4320e3a2b05daeceb06" gracePeriod=45
Jul 29 21:54:19.480772 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.480748    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:54:19.539353 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.539297    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-dshm\") pod \"6348eed9-db45-40f4-ad0d-4112261502be\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") "
Jul 29 21:54:19.539353 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.539335    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-home\") pod \"6348eed9-db45-40f4-ad0d-4112261502be\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") "
Jul 29 21:54:19.539353 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.539352    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2rt6\" (UniqueName: \"kubernetes.io/projected/6348eed9-db45-40f4-ad0d-4112261502be-kube-api-access-b2rt6\") pod \"6348eed9-db45-40f4-ad0d-4112261502be\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") "
Jul 29 21:54:19.539593 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.539379    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6348eed9-db45-40f4-ad0d-4112261502be-tls-certs\") pod \"6348eed9-db45-40f4-ad0d-4112261502be\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") "
Jul 29 21:54:19.539593 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.539405    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-tmp-dir\") pod \"6348eed9-db45-40f4-ad0d-4112261502be\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") "
Jul 29 21:54:19.539593 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.539468    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-model-cache\") pod \"6348eed9-db45-40f4-ad0d-4112261502be\" (UID: \"6348eed9-db45-40f4-ad0d-4112261502be\") "
Jul 29 21:54:19.539593 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.539552    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-home" (OuterVolumeSpecName: "home") pod "6348eed9-db45-40f4-ad0d-4112261502be" (UID: "6348eed9-db45-40f4-ad0d-4112261502be"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:54:19.539809 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.539685    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6348eed9-db45-40f4-ad0d-4112261502be" (UID: "6348eed9-db45-40f4-ad0d-4112261502be"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:54:19.539809 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.539759    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-home\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:54:19.539809 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.539771    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-model-cache" (OuterVolumeSpecName: "model-cache") pod "6348eed9-db45-40f4-ad0d-4112261502be" (UID: "6348eed9-db45-40f4-ad0d-4112261502be"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:54:19.541570 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.541548    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6348eed9-db45-40f4-ad0d-4112261502be-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6348eed9-db45-40f4-ad0d-4112261502be" (UID: "6348eed9-db45-40f4-ad0d-4112261502be"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:54:19.541663 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.541548    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6348eed9-db45-40f4-ad0d-4112261502be-kube-api-access-b2rt6" (OuterVolumeSpecName: "kube-api-access-b2rt6") pod "6348eed9-db45-40f4-ad0d-4112261502be" (UID: "6348eed9-db45-40f4-ad0d-4112261502be"). InnerVolumeSpecName "kube-api-access-b2rt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:54:19.541663 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.541551    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-dshm" (OuterVolumeSpecName: "dshm") pod "6348eed9-db45-40f4-ad0d-4112261502be" (UID: "6348eed9-db45-40f4-ad0d-4112261502be"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:54:19.640572 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.640537    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-dshm\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:54:19.640572 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.640569    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2rt6\" (UniqueName: \"kubernetes.io/projected/6348eed9-db45-40f4-ad0d-4112261502be-kube-api-access-b2rt6\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:54:19.640572 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.640580    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6348eed9-db45-40f4-ad0d-4112261502be-tls-certs\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:54:19.640751 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.640589    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-tmp-dir\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:54:19.640751 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:19.640596    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6348eed9-db45-40f4-ad0d-4112261502be-model-cache\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:54:20.035207 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.035171    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl" podUID="02d5d2f0-6588-4288-9635-8237a9391bb8" containerName="main" containerID="cri-o://3251d745cc0807059c9ecc7c4fa1d5755d8e99a95c9102310f23a726a6e7c640" gracePeriod=45
Jul 29 21:54:20.284961 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.284936    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:54:20.289741 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.289691    2580 generic.go:356] "Generic (PLEG): container finished" podID="6348eed9-db45-40f4-ad0d-4112261502be" containerID="8ec2def8e407f23e011f3bfebfcc18c13448674430cdb4320e3a2b05daeceb06" exitCode=0
Jul 29 21:54:20.289741 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.289731    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"
Jul 29 21:54:20.289875 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.289767    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd" event={"ID":"6348eed9-db45-40f4-ad0d-4112261502be","Type":"ContainerDied","Data":"8ec2def8e407f23e011f3bfebfcc18c13448674430cdb4320e3a2b05daeceb06"}
Jul 29 21:54:20.289875 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.289809    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd" event={"ID":"6348eed9-db45-40f4-ad0d-4112261502be","Type":"ContainerDied","Data":"2eb062bfceaedcc396611e5659c09cf2a473e27bbbb71e8ae460761039a82670"}
Jul 29 21:54:20.289875 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.289834    2580 scope.go:117] "RemoveContainer" containerID="8ec2def8e407f23e011f3bfebfcc18c13448674430cdb4320e3a2b05daeceb06"
Jul 29 21:54:20.291228 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.291199    2580 generic.go:356] "Generic (PLEG): container finished" podID="02d5d2f0-6588-4288-9635-8237a9391bb8" containerID="3251d745cc0807059c9ecc7c4fa1d5755d8e99a95c9102310f23a726a6e7c640" exitCode=0
Jul 29 21:54:20.291328 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.291253    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"
Jul 29 21:54:20.291328 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.291260    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl" event={"ID":"02d5d2f0-6588-4288-9635-8237a9391bb8","Type":"ContainerDied","Data":"3251d745cc0807059c9ecc7c4fa1d5755d8e99a95c9102310f23a726a6e7c640"}
Jul 29 21:54:20.291328 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.291296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl" event={"ID":"02d5d2f0-6588-4288-9635-8237a9391bb8","Type":"ContainerDied","Data":"faf1efebe7a4a0c432d6d746fbdbcf370395068e0f5b2458790a41a543f7b0fa"}
Jul 29 21:54:20.297925 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.297908    2580 scope.go:117] "RemoveContainer" containerID="8ec2def8e407f23e011f3bfebfcc18c13448674430cdb4320e3a2b05daeceb06"
Jul 29 21:54:20.298189 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:54:20.298168    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ec2def8e407f23e011f3bfebfcc18c13448674430cdb4320e3a2b05daeceb06\": container with ID starting with 8ec2def8e407f23e011f3bfebfcc18c13448674430cdb4320e3a2b05daeceb06 not found: ID does not exist" containerID="8ec2def8e407f23e011f3bfebfcc18c13448674430cdb4320e3a2b05daeceb06"
Jul 29 21:54:20.298268 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.298197    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ec2def8e407f23e011f3bfebfcc18c13448674430cdb4320e3a2b05daeceb06"} err="failed to get container status \"8ec2def8e407f23e011f3bfebfcc18c13448674430cdb4320e3a2b05daeceb06\": rpc error: code = NotFound desc = could not find container \"8ec2def8e407f23e011f3bfebfcc18c13448674430cdb4320e3a2b05daeceb06\": container with ID starting with 8ec2def8e407f23e011f3bfebfcc18c13448674430cdb4320e3a2b05daeceb06 not found: ID does not exist"
Jul 29 21:54:20.298268 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.298214    2580 scope.go:117] "RemoveContainer" containerID="3251d745cc0807059c9ecc7c4fa1d5755d8e99a95c9102310f23a726a6e7c640"
Jul 29 21:54:20.306025 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.306009    2580 scope.go:117] "RemoveContainer" containerID="3251d745cc0807059c9ecc7c4fa1d5755d8e99a95c9102310f23a726a6e7c640"
Jul 29 21:54:20.306250 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:54:20.306235    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3251d745cc0807059c9ecc7c4fa1d5755d8e99a95c9102310f23a726a6e7c640\": container with ID starting with 3251d745cc0807059c9ecc7c4fa1d5755d8e99a95c9102310f23a726a6e7c640 not found: ID does not exist" containerID="3251d745cc0807059c9ecc7c4fa1d5755d8e99a95c9102310f23a726a6e7c640"
Jul 29 21:54:20.306298 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.306255    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3251d745cc0807059c9ecc7c4fa1d5755d8e99a95c9102310f23a726a6e7c640"} err="failed to get container status \"3251d745cc0807059c9ecc7c4fa1d5755d8e99a95c9102310f23a726a6e7c640\": rpc error: code = NotFound desc = could not find container \"3251d745cc0807059c9ecc7c4fa1d5755d8e99a95c9102310f23a726a6e7c640\": container with ID starting with 3251d745cc0807059c9ecc7c4fa1d5755d8e99a95c9102310f23a726a6e7c640 not found: ID does not exist"
Jul 29 21:54:20.338235 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.338211    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"]
Jul 29 21:54:20.343305 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.343287    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-fe32bc/auth-test-model-fe32bc-kserve-7fcc9466c-x2wfd"]
Jul 29 21:54:20.346232 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.346217    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-model-cache\") pod \"02d5d2f0-6588-4288-9635-8237a9391bb8\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") "
Jul 29 21:54:20.346288 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.346245    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02d5d2f0-6588-4288-9635-8237a9391bb8-tls-certs\") pod \"02d5d2f0-6588-4288-9635-8237a9391bb8\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") "
Jul 29 21:54:20.346288 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.346263    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-tmp-dir\") pod \"02d5d2f0-6588-4288-9635-8237a9391bb8\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") "
Jul 29 21:54:20.346358 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.346304    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-dshm\") pod \"02d5d2f0-6588-4288-9635-8237a9391bb8\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") "
Jul 29 21:54:20.346358 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.346325    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-home\") pod \"02d5d2f0-6588-4288-9635-8237a9391bb8\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") "
Jul 29 21:54:20.346429 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.346360    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qfl8\" (UniqueName: \"kubernetes.io/projected/02d5d2f0-6588-4288-9635-8237a9391bb8-kube-api-access-6qfl8\") pod \"02d5d2f0-6588-4288-9635-8237a9391bb8\" (UID: \"02d5d2f0-6588-4288-9635-8237a9391bb8\") "
Jul 29 21:54:20.346518 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.346494    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-model-cache" (OuterVolumeSpecName: "model-cache") pod "02d5d2f0-6588-4288-9635-8237a9391bb8" (UID: "02d5d2f0-6588-4288-9635-8237a9391bb8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:54:20.346585 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.346523    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "02d5d2f0-6588-4288-9635-8237a9391bb8" (UID: "02d5d2f0-6588-4288-9635-8237a9391bb8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:54:20.346643 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.346586    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-home" (OuterVolumeSpecName: "home") pod "02d5d2f0-6588-4288-9635-8237a9391bb8" (UID: "02d5d2f0-6588-4288-9635-8237a9391bb8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:54:20.346698 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.346660    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-model-cache\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:54:20.346698 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.346673    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-tmp-dir\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:54:20.346698 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.346681    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-home\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:54:20.348429 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.348408    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-dshm" (OuterVolumeSpecName: "dshm") pod "02d5d2f0-6588-4288-9635-8237a9391bb8" (UID: "02d5d2f0-6588-4288-9635-8237a9391bb8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:54:20.348506 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.348466    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02d5d2f0-6588-4288-9635-8237a9391bb8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "02d5d2f0-6588-4288-9635-8237a9391bb8" (UID: "02d5d2f0-6588-4288-9635-8237a9391bb8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:54:20.348670 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.348652    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02d5d2f0-6588-4288-9635-8237a9391bb8-kube-api-access-6qfl8" (OuterVolumeSpecName: "kube-api-access-6qfl8") pod "02d5d2f0-6588-4288-9635-8237a9391bb8" (UID: "02d5d2f0-6588-4288-9635-8237a9391bb8"). InnerVolumeSpecName "kube-api-access-6qfl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:54:20.447443 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.447412    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02d5d2f0-6588-4288-9635-8237a9391bb8-dshm\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:54:20.447443 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.447440    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qfl8\" (UniqueName: \"kubernetes.io/projected/02d5d2f0-6588-4288-9635-8237a9391bb8-kube-api-access-6qfl8\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:54:20.447443 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.447450    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02d5d2f0-6588-4288-9635-8237a9391bb8-tls-certs\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:54:20.619641 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.619615    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"]
Jul 29 21:54:20.622020 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.621999    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/auth-test-model-670ddc-kserve-6bdcc85b75-bxvbl"]
Jul 29 21:54:20.715197 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.715175    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02d5d2f0-6588-4288-9635-8237a9391bb8" path="/var/lib/kubelet/pods/02d5d2f0-6588-4288-9635-8237a9391bb8/volumes"
Jul 29 21:54:20.715563 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:20.715548    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6348eed9-db45-40f4-ad0d-4112261502be" path="/var/lib/kubelet/pods/6348eed9-db45-40f4-ad0d-4112261502be/volumes"
Jul 29 21:54:31.709153 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.709114    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"]
Jul 29 21:54:31.709645 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.709459    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6348eed9-db45-40f4-ad0d-4112261502be" containerName="main"
Jul 29 21:54:31.709645 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.709471    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6348eed9-db45-40f4-ad0d-4112261502be" containerName="main"
Jul 29 21:54:31.709645 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.709496    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02d5d2f0-6588-4288-9635-8237a9391bb8" containerName="main"
Jul 29 21:54:31.709645 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.709502    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="02d5d2f0-6588-4288-9635-8237a9391bb8" containerName="main"
Jul 29 21:54:31.709645 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.709558    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6348eed9-db45-40f4-ad0d-4112261502be" containerName="main"
Jul 29 21:54:31.709645 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.709567    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="02d5d2f0-6588-4288-9635-8237a9391bb8" containerName="main"
Jul 29 21:54:31.714395 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.714369    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.717617 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.717586    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fe32bc\"/\"openshift-service-ca.crt\""
Jul 29 21:54:31.717804 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.717586    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fe32bc\"/\"default-dockercfg-7gwbz\""
Jul 29 21:54:31.718743 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.718719    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fe32bc\"/\"kube-root-ca.crt\""
Jul 29 21:54:31.718903 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.718726    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fe32bc\"/\"sub-test-model-fe32bc-kserve-self-signed-certs\""
Jul 29 21:54:31.725556 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.725523    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"]
Jul 29 21:54:31.740563 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.740518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49sdb\" (UniqueName: \"kubernetes.io/projected/cd058fd7-001a-4fce-9512-df3746f4ffbe-kube-api-access-49sdb\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.740781 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.740630    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-dshm\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.740781 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.740653    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-tmp-dir\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.740781 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.740695    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd058fd7-001a-4fce-9512-df3746f4ffbe-tls-certs\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.740781 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.740712    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-model-cache\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.740781 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.740754    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-home\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.841734 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.841693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-dshm\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.841734 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.841732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-tmp-dir\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.842107 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.841763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd058fd7-001a-4fce-9512-df3746f4ffbe-tls-certs\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.842107 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.841883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-model-cache\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.842107 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.841942    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-home\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.842107 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.842001    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49sdb\" (UniqueName: \"kubernetes.io/projected/cd058fd7-001a-4fce-9512-df3746f4ffbe-kube-api-access-49sdb\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.842301 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.842285    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-tmp-dir\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.842355 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.842310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-model-cache\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.842404 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.842351    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-home\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.844347 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.844315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-dshm\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.844749 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.844728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd058fd7-001a-4fce-9512-df3746f4ffbe-tls-certs\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:31.855215 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:31.855179    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49sdb\" (UniqueName: \"kubernetes.io/projected/cd058fd7-001a-4fce-9512-df3746f4ffbe-kube-api-access-49sdb\") pod \"sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") " pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:32.027773 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:32.027664    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:32.188582 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:32.188554    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"]
Jul 29 21:54:32.191215 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:54:32.191185    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd058fd7_001a_4fce_9512_df3746f4ffbe.slice/crio-0c2685c7599979132b75119799046be82fa2fceaa92e9dc6c2de7c7ac1327dc1 WatchSource:0}: Error finding container 0c2685c7599979132b75119799046be82fa2fceaa92e9dc6c2de7c7ac1327dc1: Status 404 returned error can't find the container with id 0c2685c7599979132b75119799046be82fa2fceaa92e9dc6c2de7c7ac1327dc1
Jul 29 21:54:32.355778 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:32.355674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx" event={"ID":"cd058fd7-001a-4fce-9512-df3746f4ffbe","Type":"ContainerStarted","Data":"9defacae2e1555081d7bbd9be5ea54897c14f472d06f19c9aeaa666c286d8f8c"}
Jul 29 21:54:32.355778 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:32.355729    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx" event={"ID":"cd058fd7-001a-4fce-9512-df3746f4ffbe","Type":"ContainerStarted","Data":"0c2685c7599979132b75119799046be82fa2fceaa92e9dc6c2de7c7ac1327dc1"}
Jul 29 21:54:32.375541 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:32.375419    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx" podStartSLOduration=1.375392643 podStartE2EDuration="1.375392643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:54:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:54:32.374161057 +0000 UTC m=+3024.255435211" watchObservedRunningTime="2026-07-29 21:54:32.375392643 +0000 UTC m=+3024.256666783"
Jul 29 21:54:42.028323 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:42.028277    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:42.028920 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:42.028340    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:42.040759 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:42.040728    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:42.414109 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:42.414078    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:54:50.162052 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.162016    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"]
Jul 29 21:54:50.165221 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.165204    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.172795 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.172774    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-670ddc\"/\"kube-root-ca.crt\""
Jul 29 21:54:50.172937 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.172790    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-670ddc\"/\"default-dockercfg-82w8b\""
Jul 29 21:54:50.172937 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.172833    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-670ddc\"/\"openshift-service-ca.crt\""
Jul 29 21:54:50.172937 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.172892    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-670ddc\"/\"sub-test-model-670ddc-kserve-self-signed-certs\""
Jul 29 21:54:50.189079 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.189054    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"]
Jul 29 21:54:50.292196 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.292157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-tmp-dir\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.292321 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.292211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-dshm\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.292321 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.292262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghbgk\" (UniqueName: \"kubernetes.io/projected/957ee42f-0a94-4d55-b729-aa0eba97d813-kube-api-access-ghbgk\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.292321 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.292296    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-model-cache\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.292438 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.292323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/957ee42f-0a94-4d55-b729-aa0eba97d813-tls-certs\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.292438 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.292357    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-home\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.393044 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.393017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-dshm\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.393171 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.393052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghbgk\" (UniqueName: \"kubernetes.io/projected/957ee42f-0a94-4d55-b729-aa0eba97d813-kube-api-access-ghbgk\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.393171 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.393072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-model-cache\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.393171 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.393095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/957ee42f-0a94-4d55-b729-aa0eba97d813-tls-certs\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.393337 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.393214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-home\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.393337 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.393286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-tmp-dir\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.393520 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.393494    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-model-cache\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.393580 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.393544    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-home\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.393644 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.393622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-tmp-dir\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.395406 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.395385    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-dshm\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.395696 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.395676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/957ee42f-0a94-4d55-b729-aa0eba97d813-tls-certs\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.406073 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.406048    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghbgk\" (UniqueName: \"kubernetes.io/projected/957ee42f-0a94-4d55-b729-aa0eba97d813-kube-api-access-ghbgk\") pod \"sub-test-model-670ddc-kserve-554c4bddc5-89nxj\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") " pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.474934 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.474888    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:54:50.607358 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:50.607334    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"]
Jul 29 21:54:50.608814 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:54:50.608792    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod957ee42f_0a94_4d55_b729_aa0eba97d813.slice/crio-45c49ca059ab40f6c4208e2c63fe8206906bf4bff4b2fe62df5df9e09e24afe2 WatchSource:0}: Error finding container 45c49ca059ab40f6c4208e2c63fe8206906bf4bff4b2fe62df5df9e09e24afe2: Status 404 returned error can't find the container with id 45c49ca059ab40f6c4208e2c63fe8206906bf4bff4b2fe62df5df9e09e24afe2
Jul 29 21:54:51.448091 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:51.448051    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj" event={"ID":"957ee42f-0a94-4d55-b729-aa0eba97d813","Type":"ContainerStarted","Data":"5beff86fca735c7b9d22b325350c4c06899c58a5f19710abb0867eb46ea443ed"}
Jul 29 21:54:51.448463 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:51.448096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj" event={"ID":"957ee42f-0a94-4d55-b729-aa0eba97d813","Type":"ContainerStarted","Data":"45c49ca059ab40f6c4208e2c63fe8206906bf4bff4b2fe62df5df9e09e24afe2"}
Jul 29 21:54:51.476319 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:54:51.476268    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj" podStartSLOduration=1.4762536609999999 podStartE2EDuration="1.476253661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:54:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:54:51.474680613 +0000 UTC m=+3043.355954771" watchObservedRunningTime="2026-07-29 21:54:51.476253661 +0000 UTC m=+3043.357527798"
Jul 29 21:55:00.475471 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:55:00.475369    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:55:00.475471 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:55:00.475450    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:55:00.488511 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:55:00.488483    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:55:00.502378 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:55:00.502358    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:56:22.715954 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:22.715911    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"]
Jul 29 21:56:23.604406 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:23.604374    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"]
Jul 29 21:56:37.733882 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:37.733795    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx" podUID="cd058fd7-001a-4fce-9512-df3746f4ffbe" containerName="main" containerID="cri-o://9defacae2e1555081d7bbd9be5ea54897c14f472d06f19c9aeaa666c286d8f8c" gracePeriod=45
Jul 29 21:56:37.949535 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:37.949497    2580 generic.go:356] "Generic (PLEG): container finished" podID="cd058fd7-001a-4fce-9512-df3746f4ffbe" containerID="9defacae2e1555081d7bbd9be5ea54897c14f472d06f19c9aeaa666c286d8f8c" exitCode=0
Jul 29 21:56:37.949670 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:37.949568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx" event={"ID":"cd058fd7-001a-4fce-9512-df3746f4ffbe","Type":"ContainerDied","Data":"9defacae2e1555081d7bbd9be5ea54897c14f472d06f19c9aeaa666c286d8f8c"}
Jul 29 21:56:37.976819 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:37.976802    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:56:38.115786 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.115735    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49sdb\" (UniqueName: \"kubernetes.io/projected/cd058fd7-001a-4fce-9512-df3746f4ffbe-kube-api-access-49sdb\") pod \"cd058fd7-001a-4fce-9512-df3746f4ffbe\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") "
Jul 29 21:56:38.115786 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.115770    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-model-cache\") pod \"cd058fd7-001a-4fce-9512-df3746f4ffbe\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") "
Jul 29 21:56:38.115933 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.115805    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-dshm\") pod \"cd058fd7-001a-4fce-9512-df3746f4ffbe\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") "
Jul 29 21:56:38.115933 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.115880    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd058fd7-001a-4fce-9512-df3746f4ffbe-tls-certs\") pod \"cd058fd7-001a-4fce-9512-df3746f4ffbe\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") "
Jul 29 21:56:38.115933 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.115904    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-tmp-dir\") pod \"cd058fd7-001a-4fce-9512-df3746f4ffbe\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") "
Jul 29 21:56:38.115933 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.115924    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-home\") pod \"cd058fd7-001a-4fce-9512-df3746f4ffbe\" (UID: \"cd058fd7-001a-4fce-9512-df3746f4ffbe\") "
Jul 29 21:56:38.116122 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.116040    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd058fd7-001a-4fce-9512-df3746f4ffbe" (UID: "cd058fd7-001a-4fce-9512-df3746f4ffbe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:56:38.116225 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.116205    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-model-cache\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:56:38.116225 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.116201    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd058fd7-001a-4fce-9512-df3746f4ffbe" (UID: "cd058fd7-001a-4fce-9512-df3746f4ffbe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:56:38.116363 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.116241    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-home" (OuterVolumeSpecName: "home") pod "cd058fd7-001a-4fce-9512-df3746f4ffbe" (UID: "cd058fd7-001a-4fce-9512-df3746f4ffbe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:56:38.117919 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.117896    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-dshm" (OuterVolumeSpecName: "dshm") pod "cd058fd7-001a-4fce-9512-df3746f4ffbe" (UID: "cd058fd7-001a-4fce-9512-df3746f4ffbe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:56:38.118013 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.117958    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd058fd7-001a-4fce-9512-df3746f4ffbe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd058fd7-001a-4fce-9512-df3746f4ffbe" (UID: "cd058fd7-001a-4fce-9512-df3746f4ffbe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:56:38.118068 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.118051    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd058fd7-001a-4fce-9512-df3746f4ffbe-kube-api-access-49sdb" (OuterVolumeSpecName: "kube-api-access-49sdb") pod "cd058fd7-001a-4fce-9512-df3746f4ffbe" (UID: "cd058fd7-001a-4fce-9512-df3746f4ffbe"). InnerVolumeSpecName "kube-api-access-49sdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:56:38.217493 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.217464    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd058fd7-001a-4fce-9512-df3746f4ffbe-tls-certs\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:56:38.217493 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.217492    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-tmp-dir\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:56:38.217617 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.217505    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-home\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:56:38.217617 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.217517    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49sdb\" (UniqueName: \"kubernetes.io/projected/cd058fd7-001a-4fce-9512-df3746f4ffbe-kube-api-access-49sdb\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:56:38.217617 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.217528    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd058fd7-001a-4fce-9512-df3746f4ffbe-dshm\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:56:38.622231 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.622177    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj" podUID="957ee42f-0a94-4d55-b729-aa0eba97d813" containerName="main" containerID="cri-o://5beff86fca735c7b9d22b325350c4c06899c58a5f19710abb0867eb46ea443ed" gracePeriod=45
Jul 29 21:56:38.872202 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.872181    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:56:38.955593 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.955568    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"
Jul 29 21:56:38.955593 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.955585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx" event={"ID":"cd058fd7-001a-4fce-9512-df3746f4ffbe","Type":"ContainerDied","Data":"0c2685c7599979132b75119799046be82fa2fceaa92e9dc6c2de7c7ac1327dc1"}
Jul 29 21:56:38.955776 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.955635    2580 scope.go:117] "RemoveContainer" containerID="9defacae2e1555081d7bbd9be5ea54897c14f472d06f19c9aeaa666c286d8f8c"
Jul 29 21:56:38.956816 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.956792    2580 generic.go:356] "Generic (PLEG): container finished" podID="957ee42f-0a94-4d55-b729-aa0eba97d813" containerID="5beff86fca735c7b9d22b325350c4c06899c58a5f19710abb0867eb46ea443ed" exitCode=0
Jul 29 21:56:38.956937 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.956824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj" event={"ID":"957ee42f-0a94-4d55-b729-aa0eba97d813","Type":"ContainerDied","Data":"5beff86fca735c7b9d22b325350c4c06899c58a5f19710abb0867eb46ea443ed"}
Jul 29 21:56:38.956937 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.956859    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"
Jul 29 21:56:38.956937 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.956879    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj" event={"ID":"957ee42f-0a94-4d55-b729-aa0eba97d813","Type":"ContainerDied","Data":"45c49ca059ab40f6c4208e2c63fe8206906bf4bff4b2fe62df5df9e09e24afe2"}
Jul 29 21:56:38.963689 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.963669    2580 scope.go:117] "RemoveContainer" containerID="5beff86fca735c7b9d22b325350c4c06899c58a5f19710abb0867eb46ea443ed"
Jul 29 21:56:38.972889 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.972873    2580 scope.go:117] "RemoveContainer" containerID="5beff86fca735c7b9d22b325350c4c06899c58a5f19710abb0867eb46ea443ed"
Jul 29 21:56:38.973139 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:56:38.973123    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5beff86fca735c7b9d22b325350c4c06899c58a5f19710abb0867eb46ea443ed\": container with ID starting with 5beff86fca735c7b9d22b325350c4c06899c58a5f19710abb0867eb46ea443ed not found: ID does not exist" containerID="5beff86fca735c7b9d22b325350c4c06899c58a5f19710abb0867eb46ea443ed"
Jul 29 21:56:38.973198 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.973145    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5beff86fca735c7b9d22b325350c4c06899c58a5f19710abb0867eb46ea443ed"} err="failed to get container status \"5beff86fca735c7b9d22b325350c4c06899c58a5f19710abb0867eb46ea443ed\": rpc error: code = NotFound desc = could not find container \"5beff86fca735c7b9d22b325350c4c06899c58a5f19710abb0867eb46ea443ed\": container with ID starting with 5beff86fca735c7b9d22b325350c4c06899c58a5f19710abb0867eb46ea443ed not found: ID does not exist"
Jul 29 21:56:38.979169 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.979149    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"]
Jul 29 21:56:38.983375 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:38.983356    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-fe32bc/sub-test-model-fe32bc-kserve-65f6b7dd5-5krnx"]
Jul 29 21:56:39.023817 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.023795    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-dshm\") pod \"957ee42f-0a94-4d55-b729-aa0eba97d813\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") "
Jul 29 21:56:39.023933 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.023820    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-model-cache\") pod \"957ee42f-0a94-4d55-b729-aa0eba97d813\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") "
Jul 29 21:56:39.023933 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.023868    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-home\") pod \"957ee42f-0a94-4d55-b729-aa0eba97d813\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") "
Jul 29 21:56:39.023933 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.023907    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-tmp-dir\") pod \"957ee42f-0a94-4d55-b729-aa0eba97d813\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") "
Jul 29 21:56:39.023933 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.023931    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/957ee42f-0a94-4d55-b729-aa0eba97d813-tls-certs\") pod \"957ee42f-0a94-4d55-b729-aa0eba97d813\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") "
Jul 29 21:56:39.024142 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.023959    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghbgk\" (UniqueName: \"kubernetes.io/projected/957ee42f-0a94-4d55-b729-aa0eba97d813-kube-api-access-ghbgk\") pod \"957ee42f-0a94-4d55-b729-aa0eba97d813\" (UID: \"957ee42f-0a94-4d55-b729-aa0eba97d813\") "
Jul 29 21:56:39.024142 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.024066    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-model-cache" (OuterVolumeSpecName: "model-cache") pod "957ee42f-0a94-4d55-b729-aa0eba97d813" (UID: "957ee42f-0a94-4d55-b729-aa0eba97d813"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:56:39.024214 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.024140    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "957ee42f-0a94-4d55-b729-aa0eba97d813" (UID: "957ee42f-0a94-4d55-b729-aa0eba97d813"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:56:39.024214 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.024155    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-model-cache\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:56:39.024214 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.024158    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-home" (OuterVolumeSpecName: "home") pod "957ee42f-0a94-4d55-b729-aa0eba97d813" (UID: "957ee42f-0a94-4d55-b729-aa0eba97d813"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:56:39.025818 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.025794    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-dshm" (OuterVolumeSpecName: "dshm") pod "957ee42f-0a94-4d55-b729-aa0eba97d813" (UID: "957ee42f-0a94-4d55-b729-aa0eba97d813"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:56:39.025818 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.025806    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/957ee42f-0a94-4d55-b729-aa0eba97d813-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "957ee42f-0a94-4d55-b729-aa0eba97d813" (UID: "957ee42f-0a94-4d55-b729-aa0eba97d813"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:56:39.025984 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.025805    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/957ee42f-0a94-4d55-b729-aa0eba97d813-kube-api-access-ghbgk" (OuterVolumeSpecName: "kube-api-access-ghbgk") pod "957ee42f-0a94-4d55-b729-aa0eba97d813" (UID: "957ee42f-0a94-4d55-b729-aa0eba97d813"). InnerVolumeSpecName "kube-api-access-ghbgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:56:39.125171 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.125148    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/957ee42f-0a94-4d55-b729-aa0eba97d813-tls-certs\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:56:39.125270 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.125175    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghbgk\" (UniqueName: \"kubernetes.io/projected/957ee42f-0a94-4d55-b729-aa0eba97d813-kube-api-access-ghbgk\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:56:39.125270 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.125187    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-dshm\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:56:39.125270 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.125198    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-home\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:56:39.125270 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.125210    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/957ee42f-0a94-4d55-b729-aa0eba97d813-tmp-dir\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:56:39.279672 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.279641    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"]
Jul 29 21:56:39.285201 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:39.285177    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/sub-test-model-670ddc-kserve-554c4bddc5-89nxj"]
Jul 29 21:56:40.715228 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:40.715178    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="957ee42f-0a94-4d55-b729-aa0eba97d813" path="/var/lib/kubelet/pods/957ee42f-0a94-4d55-b729-aa0eba97d813/volumes"
Jul 29 21:56:40.715788 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:56:40.715765    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd058fd7-001a-4fce-9512-df3746f4ffbe" path="/var/lib/kubelet/pods/cd058fd7-001a-4fce-9512-df3746f4ffbe/volumes"
Jul 29 21:57:15.577480 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.577443    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"]
Jul 29 21:57:15.578055 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.577941    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd058fd7-001a-4fce-9512-df3746f4ffbe" containerName="main"
Jul 29 21:57:15.578055 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.577962    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd058fd7-001a-4fce-9512-df3746f4ffbe" containerName="main"
Jul 29 21:57:15.578055 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.577994    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="957ee42f-0a94-4d55-b729-aa0eba97d813" containerName="main"
Jul 29 21:57:15.578055 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.578003    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="957ee42f-0a94-4d55-b729-aa0eba97d813" containerName="main"
Jul 29 21:57:15.578218 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.578075    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd058fd7-001a-4fce-9512-df3746f4ffbe" containerName="main"
Jul 29 21:57:15.578218 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.578084    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="957ee42f-0a94-4d55-b729-aa0eba97d813" containerName="main"
Jul 29 21:57:15.582338 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.582312    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.586766 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.586743    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-670ddc\"/\"kube-root-ca.crt\""
Jul 29 21:57:15.586766 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.586762    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-670ddc\"/\"openshift-service-ca.crt\""
Jul 29 21:57:15.586959 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.586745    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-670ddc\"/\"rate-test-model-670ddc-kserve-self-signed-certs\""
Jul 29 21:57:15.586959 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.586746    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-670ddc\"/\"default-dockercfg-82w8b\""
Jul 29 21:57:15.590558 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.590514    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"]
Jul 29 21:57:15.694001 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.693981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-model-cache\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.694099 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.694013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ecdf552-dae7-4790-95fd-5f96c72b2aad-tls-certs\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.694099 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.694049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-tmp-dir\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.694172 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.694115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdgp5\" (UniqueName: \"kubernetes.io/projected/2ecdf552-dae7-4790-95fd-5f96c72b2aad-kube-api-access-sdgp5\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.694172 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.694148    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-home\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.694233 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.694174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-dshm\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.795307 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.795285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-model-cache\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.795412 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.795318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ecdf552-dae7-4790-95fd-5f96c72b2aad-tls-certs\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.795412 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.795350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-tmp-dir\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.795412 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.795379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdgp5\" (UniqueName: \"kubernetes.io/projected/2ecdf552-dae7-4790-95fd-5f96c72b2aad-kube-api-access-sdgp5\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.795412 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.795400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-home\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.795547 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.795421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-dshm\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.795734 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.795714    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-model-cache\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.795804 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.795779    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-home\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.795958 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.795834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-tmp-dir\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.797577 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.797558    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-dshm\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.797866 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.797830    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ecdf552-dae7-4790-95fd-5f96c72b2aad-tls-certs\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.805886 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.805827    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdgp5\" (UniqueName: \"kubernetes.io/projected/2ecdf552-dae7-4790-95fd-5f96c72b2aad-kube-api-access-sdgp5\") pod \"rate-test-model-670ddc-kserve-794d6ff459-5t5pw\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") " pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:15.892649 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:15.892617    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:16.024793 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:16.024759    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"]
Jul 29 21:57:16.028516 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:57:16.028490    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ecdf552_dae7_4790_95fd_5f96c72b2aad.slice/crio-6b4d9acb44cecc4afd649d36572e7c4a2bf60b9fe3b4a7b7ce29e8d38c31bda4 WatchSource:0}: Error finding container 6b4d9acb44cecc4afd649d36572e7c4a2bf60b9fe3b4a7b7ce29e8d38c31bda4: Status 404 returned error can't find the container with id 6b4d9acb44cecc4afd649d36572e7c4a2bf60b9fe3b4a7b7ce29e8d38c31bda4
Jul 29 21:57:16.130150 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:16.130122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw" event={"ID":"2ecdf552-dae7-4790-95fd-5f96c72b2aad","Type":"ContainerStarted","Data":"16175bd1e6f0bef47e76c721b661e7e7088094142d2cb1fa14f5257a2e0daeb6"}
Jul 29 21:57:16.130255 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:16.130164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw" event={"ID":"2ecdf552-dae7-4790-95fd-5f96c72b2aad","Type":"ContainerStarted","Data":"6b4d9acb44cecc4afd649d36572e7c4a2bf60b9fe3b4a7b7ce29e8d38c31bda4"}
Jul 29 21:57:16.149372 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:16.149290    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw" podStartSLOduration=1.149273108 podStartE2EDuration="1.149273108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:57:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:57:16.148898117 +0000 UTC m=+3188.030172254" watchObservedRunningTime="2026-07-29 21:57:16.149273108 +0000 UTC m=+3188.030547246"
Jul 29 21:57:25.893761 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:25.893719    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:25.894296 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:25.893775    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:25.906203 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:25.906174    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:57:26.188187 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:57:26.188107    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:58:16.231040 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:16.230967    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"]
Jul 29 21:58:31.248664 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.248618    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw" podUID="2ecdf552-dae7-4790-95fd-5f96c72b2aad" containerName="main" containerID="cri-o://16175bd1e6f0bef47e76c721b661e7e7088094142d2cb1fa14f5257a2e0daeb6" gracePeriod=45
Jul 29 21:58:31.486545 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.486518    2580 generic.go:356] "Generic (PLEG): container finished" podID="2ecdf552-dae7-4790-95fd-5f96c72b2aad" containerID="16175bd1e6f0bef47e76c721b661e7e7088094142d2cb1fa14f5257a2e0daeb6" exitCode=0
Jul 29 21:58:31.486677 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.486551    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw" event={"ID":"2ecdf552-dae7-4790-95fd-5f96c72b2aad","Type":"ContainerDied","Data":"16175bd1e6f0bef47e76c721b661e7e7088094142d2cb1fa14f5257a2e0daeb6"}
Jul 29 21:58:31.486677 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.486594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw" event={"ID":"2ecdf552-dae7-4790-95fd-5f96c72b2aad","Type":"ContainerDied","Data":"6b4d9acb44cecc4afd649d36572e7c4a2bf60b9fe3b4a7b7ce29e8d38c31bda4"}
Jul 29 21:58:31.486677 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.486605    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b4d9acb44cecc4afd649d36572e7c4a2bf60b9fe3b4a7b7ce29e8d38c31bda4"
Jul 29 21:58:31.497172 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.497155    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:58:31.554519 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.554466    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-model-cache\") pod \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") "
Jul 29 21:58:31.554519 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.554498    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-tmp-dir\") pod \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") "
Jul 29 21:58:31.554519 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.554514    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-home\") pod \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") "
Jul 29 21:58:31.554669 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.554580    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdgp5\" (UniqueName: \"kubernetes.io/projected/2ecdf552-dae7-4790-95fd-5f96c72b2aad-kube-api-access-sdgp5\") pod \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") "
Jul 29 21:58:31.554669 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.554644    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-dshm\") pod \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") "
Jul 29 21:58:31.554758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.554692    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ecdf552-dae7-4790-95fd-5f96c72b2aad-tls-certs\") pod \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\" (UID: \"2ecdf552-dae7-4790-95fd-5f96c72b2aad\") "
Jul 29 21:58:31.554758 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.554747    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-model-cache" (OuterVolumeSpecName: "model-cache") pod "2ecdf552-dae7-4790-95fd-5f96c72b2aad" (UID: "2ecdf552-dae7-4790-95fd-5f96c72b2aad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:58:31.554877 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.554762    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2ecdf552-dae7-4790-95fd-5f96c72b2aad" (UID: "2ecdf552-dae7-4790-95fd-5f96c72b2aad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:58:31.554877 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.554819    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-home" (OuterVolumeSpecName: "home") pod "2ecdf552-dae7-4790-95fd-5f96c72b2aad" (UID: "2ecdf552-dae7-4790-95fd-5f96c72b2aad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:58:31.554998 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.554960    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-model-cache\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:58:31.554998 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.554976    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-tmp-dir\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:58:31.554998 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.554985    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-home\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:58:31.556645 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.556622    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ecdf552-dae7-4790-95fd-5f96c72b2aad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2ecdf552-dae7-4790-95fd-5f96c72b2aad" (UID: "2ecdf552-dae7-4790-95fd-5f96c72b2aad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:58:31.556726 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.556660    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ecdf552-dae7-4790-95fd-5f96c72b2aad-kube-api-access-sdgp5" (OuterVolumeSpecName: "kube-api-access-sdgp5") pod "2ecdf552-dae7-4790-95fd-5f96c72b2aad" (UID: "2ecdf552-dae7-4790-95fd-5f96c72b2aad"). InnerVolumeSpecName "kube-api-access-sdgp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:58:31.556726 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.556658    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-dshm" (OuterVolumeSpecName: "dshm") pod "2ecdf552-dae7-4790-95fd-5f96c72b2aad" (UID: "2ecdf552-dae7-4790-95fd-5f96c72b2aad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:58:31.655494 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.655473    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ecdf552-dae7-4790-95fd-5f96c72b2aad-tls-certs\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:58:31.655494 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.655493    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdgp5\" (UniqueName: \"kubernetes.io/projected/2ecdf552-dae7-4790-95fd-5f96c72b2aad-kube-api-access-sdgp5\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:58:31.655618 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.655503    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ecdf552-dae7-4790-95fd-5f96c72b2aad-dshm\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:58:31.968962 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.968932    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7730-139747280796672"]
Jul 29 21:58:31.969288 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.969273    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ecdf552-dae7-4790-95fd-5f96c72b2aad" containerName="main"
Jul 29 21:58:31.969336 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.969289    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ecdf552-dae7-4790-95fd-5f96c72b2aad" containerName="main"
Jul 29 21:58:31.969371 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.969354    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ecdf552-dae7-4790-95fd-5f96c72b2aad" containerName="main"
Jul 29 21:58:31.972294 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.972278    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-139747280796672"
Jul 29 21:58:31.975285 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.975265    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rxjvh\""
Jul 29 21:58:31.979817 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:31.979790    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-139747280796672"]
Jul 29 21:58:32.059040 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:32.059014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrz8x\" (UniqueName: \"kubernetes.io/projected/ba4d79f5-920c-4d56-b110-24441561f4ab-kube-api-access-nrz8x\") pod \"test-curl-7730-139747280796672\" (UID: \"ba4d79f5-920c-4d56-b110-24441561f4ab\") " pod="opendatahub/test-curl-7730-139747280796672"
Jul 29 21:58:32.159930 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:32.159904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrz8x\" (UniqueName: \"kubernetes.io/projected/ba4d79f5-920c-4d56-b110-24441561f4ab-kube-api-access-nrz8x\") pod \"test-curl-7730-139747280796672\" (UID: \"ba4d79f5-920c-4d56-b110-24441561f4ab\") " pod="opendatahub/test-curl-7730-139747280796672"
Jul 29 21:58:32.175189 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:32.175169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrz8x\" (UniqueName: \"kubernetes.io/projected/ba4d79f5-920c-4d56-b110-24441561f4ab-kube-api-access-nrz8x\") pod \"test-curl-7730-139747280796672\" (UID: \"ba4d79f5-920c-4d56-b110-24441561f4ab\") " pod="opendatahub/test-curl-7730-139747280796672"
Jul 29 21:58:32.281728 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:32.281682    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-139747280796672"
Jul 29 21:58:32.407450 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:32.407427    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-139747280796672"]
Jul 29 21:58:32.409373 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:58:32.409349    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba4d79f5_920c_4d56_b110_24441561f4ab.slice/crio-9fd7d3ba815a4b81962bcdf339dd86206ee488eb78d221f4ddf04075716c9b20 WatchSource:0}: Error finding container 9fd7d3ba815a4b81962bcdf339dd86206ee488eb78d221f4ddf04075716c9b20: Status 404 returned error can't find the container with id 9fd7d3ba815a4b81962bcdf339dd86206ee488eb78d221f4ddf04075716c9b20
Jul 29 21:58:32.410607 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:32.410588    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:58:32.494745 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:32.494714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-139747280796672" event={"ID":"ba4d79f5-920c-4d56-b110-24441561f4ab","Type":"ContainerStarted","Data":"9fd7d3ba815a4b81962bcdf339dd86206ee488eb78d221f4ddf04075716c9b20"}
Jul 29 21:58:32.494833 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:32.494725    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"
Jul 29 21:58:32.518151 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:32.518125    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"]
Jul 29 21:58:32.522043 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:32.522021    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-670ddc/rate-test-model-670ddc-kserve-794d6ff459-5t5pw"]
Jul 29 21:58:32.715760 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:32.715736    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ecdf552-dae7-4790-95fd-5f96c72b2aad" path="/var/lib/kubelet/pods/2ecdf552-dae7-4790-95fd-5f96c72b2aad/volumes"
Jul 29 21:58:33.502830 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:33.502798    2580 generic.go:356] "Generic (PLEG): container finished" podID="ba4d79f5-920c-4d56-b110-24441561f4ab" containerID="6c5e79484373d28a525fb85af492a6319b653f144e328d06dbc52da22a089833" exitCode=0
Jul 29 21:58:33.503168 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:33.502887    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-139747280796672" event={"ID":"ba4d79f5-920c-4d56-b110-24441561f4ab","Type":"ContainerDied","Data":"6c5e79484373d28a525fb85af492a6319b653f144e328d06dbc52da22a089833"}
Jul 29 21:58:34.640628 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:34.640603    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-139747280796672"
Jul 29 21:58:34.680166 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:34.680144    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7730-139747280796672_ba4d79f5-920c-4d56-b110-24441561f4ab/test-curl-7730-139747280796672/0.log"
Jul 29 21:58:34.682303 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:34.682285    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrz8x\" (UniqueName: \"kubernetes.io/projected/ba4d79f5-920c-4d56-b110-24441561f4ab-kube-api-access-nrz8x\") pod \"ba4d79f5-920c-4d56-b110-24441561f4ab\" (UID: \"ba4d79f5-920c-4d56-b110-24441561f4ab\") "
Jul 29 21:58:34.684293 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:34.684269    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba4d79f5-920c-4d56-b110-24441561f4ab-kube-api-access-nrz8x" (OuterVolumeSpecName: "kube-api-access-nrz8x") pod "ba4d79f5-920c-4d56-b110-24441561f4ab" (UID: "ba4d79f5-920c-4d56-b110-24441561f4ab"). InnerVolumeSpecName "kube-api-access-nrz8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:58:34.702606 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:34.702585    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7730-139747280796672"]
Jul 29 21:58:34.706901 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:34.706882    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7730-139747280796672"]
Jul 29 21:58:34.714520 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:34.714499    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba4d79f5-920c-4d56-b110-24441561f4ab" path="/var/lib/kubelet/pods/ba4d79f5-920c-4d56-b110-24441561f4ab/volumes"
Jul 29 21:58:34.783632 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:34.783573    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrz8x\" (UniqueName: \"kubernetes.io/projected/ba4d79f5-920c-4d56-b110-24441561f4ab-kube-api-access-nrz8x\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 21:58:35.513743 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:35.513713    2580 scope.go:117] "RemoveContainer" containerID="6c5e79484373d28a525fb85af492a6319b653f144e328d06dbc52da22a089833"
Jul 29 21:58:35.513923 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:35.513718    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-139747280796672"
Jul 29 21:58:39.814746 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:39.814707    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"]
Jul 29 21:58:39.815251 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:39.815055    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba4d79f5-920c-4d56-b110-24441561f4ab" containerName="test-curl-7730-139747280796672"
Jul 29 21:58:39.815251 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:39.815068    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba4d79f5-920c-4d56-b110-24441561f4ab" containerName="test-curl-7730-139747280796672"
Jul 29 21:58:39.815251 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:39.815124    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba4d79f5-920c-4d56-b110-24441561f4ab" containerName="test-curl-7730-139747280796672"
Jul 29 21:58:39.819641 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:39.819621    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 21:58:39.824743 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:39.824719    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-77t9w\""
Jul 29 21:58:39.824944 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:39.824924    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-a0dedf-serving-cert\""
Jul 29 21:58:39.825098 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:39.824986    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 21:58:39.826049 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:39.826026    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 21:58:39.837999 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:39.837977    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"]
Jul 29 21:58:39.920959 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:39.920936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/920a1d5a-b13c-48c9-819a-040a450aae3b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv\" (UID: \"920a1d5a-b13c-48c9-819a-040a450aae3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 21:58:39.921072 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:39.920974    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dbv7\" (UniqueName: \"kubernetes.io/projected/920a1d5a-b13c-48c9-819a-040a450aae3b-kube-api-access-7dbv7\") pod \"maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv\" (UID: \"920a1d5a-b13c-48c9-819a-040a450aae3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 21:58:39.921072 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:39.920997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/920a1d5a-b13c-48c9-819a-040a450aae3b-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv\" (UID: \"920a1d5a-b13c-48c9-819a-040a450aae3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 21:58:40.022091 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:40.022066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dbv7\" (UniqueName: \"kubernetes.io/projected/920a1d5a-b13c-48c9-819a-040a450aae3b-kube-api-access-7dbv7\") pod \"maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv\" (UID: \"920a1d5a-b13c-48c9-819a-040a450aae3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 21:58:40.022216 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:40.022101    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/920a1d5a-b13c-48c9-819a-040a450aae3b-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv\" (UID: \"920a1d5a-b13c-48c9-819a-040a450aae3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 21:58:40.022216 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:40.022148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/920a1d5a-b13c-48c9-819a-040a450aae3b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv\" (UID: \"920a1d5a-b13c-48c9-819a-040a450aae3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 21:58:40.022833 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:40.022811    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/920a1d5a-b13c-48c9-819a-040a450aae3b-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv\" (UID: \"920a1d5a-b13c-48c9-819a-040a450aae3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 21:58:40.024526 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:40.024508    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/920a1d5a-b13c-48c9-819a-040a450aae3b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv\" (UID: \"920a1d5a-b13c-48c9-819a-040a450aae3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 21:58:40.036284 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:40.036259    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dbv7\" (UniqueName: \"kubernetes.io/projected/920a1d5a-b13c-48c9-819a-040a450aae3b-kube-api-access-7dbv7\") pod \"maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv\" (UID: \"920a1d5a-b13c-48c9-819a-040a450aae3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 21:58:40.130731 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:40.130679    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 21:58:40.264616 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:40.264593    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"]
Jul 29 21:58:40.266364 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:58:40.266337    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod920a1d5a_b13c_48c9_819a_040a450aae3b.slice/crio-3cf58e388f5fdfedada59538219c4b5e92fc2cb5d0a239d0a30c48aa7f5c8ff0 WatchSource:0}: Error finding container 3cf58e388f5fdfedada59538219c4b5e92fc2cb5d0a239d0a30c48aa7f5c8ff0: Status 404 returned error can't find the container with id 3cf58e388f5fdfedada59538219c4b5e92fc2cb5d0a239d0a30c48aa7f5c8ff0
Jul 29 21:58:40.538732 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:40.538698    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv" event={"ID":"920a1d5a-b13c-48c9-819a-040a450aae3b","Type":"ContainerStarted","Data":"3cf58e388f5fdfedada59538219c4b5e92fc2cb5d0a239d0a30c48aa7f5c8ff0"}
Jul 29 21:58:43.557065 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:43.557017    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv" event={"ID":"920a1d5a-b13c-48c9-819a-040a450aae3b","Type":"ContainerStarted","Data":"539194e6da06fab5194eaf59670a276f9002c6419d9ca0d9120e8d00f9bba120"}
Jul 29 21:58:43.557481 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:43.557239    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 21:58:43.575207 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:43.575155    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv" podStartSLOduration=2.077252055 podStartE2EDuration="4.575139445s" totalImagesPullingTime="2.49788739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:58:43.573460594 +0000 UTC m=+3275.454734751" watchObservedRunningTime="2026-07-29 21:58:43.575139445 +0000 UTC m=+3275.456413582"
Jul 29 21:58:49.567587 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:58:49.567549    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 21:59:08.798568 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:08.798539    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:59:08.803655 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:08.803279    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:59:08.810340 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:08.810319    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 21:59:08.814356 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:08.814338    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 21:59:08.935349 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:08.935323    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"]
Jul 29 21:59:08.940739 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:08.940719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:08.943584 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:08.943566    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-50d50c-serving-cert\""
Jul 29 21:59:08.948329 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:08.948308    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"]
Jul 29 21:59:09.040929 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:09.040906    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0020d846-4a8a-487b-9fe6-d03832e098b3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf\" (UID: \"0020d846-4a8a-487b-9fe6-d03832e098b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:09.041065 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:09.040953    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pqsr\" (UniqueName: \"kubernetes.io/projected/0020d846-4a8a-487b-9fe6-d03832e098b3-kube-api-access-7pqsr\") pod \"maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf\" (UID: \"0020d846-4a8a-487b-9fe6-d03832e098b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:09.041065 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:09.040987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0020d846-4a8a-487b-9fe6-d03832e098b3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf\" (UID: \"0020d846-4a8a-487b-9fe6-d03832e098b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:09.142326 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:09.142293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0020d846-4a8a-487b-9fe6-d03832e098b3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf\" (UID: \"0020d846-4a8a-487b-9fe6-d03832e098b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:09.142445 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:09.142346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pqsr\" (UniqueName: \"kubernetes.io/projected/0020d846-4a8a-487b-9fe6-d03832e098b3-kube-api-access-7pqsr\") pod \"maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf\" (UID: \"0020d846-4a8a-487b-9fe6-d03832e098b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:09.142445 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:09.142369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0020d846-4a8a-487b-9fe6-d03832e098b3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf\" (UID: \"0020d846-4a8a-487b-9fe6-d03832e098b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:09.142445 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:59:09.142435    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-serving-cert: secret "maas-api-e2e-ipp-b-50d50c-serving-cert" not found
Jul 29 21:59:09.142602 ip-10-0-138-99 kubenswrapper[2580]: E0729 21:59:09.142499    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0020d846-4a8a-487b-9fe6-d03832e098b3-maas-api-tls podName:0020d846-4a8a-487b-9fe6-d03832e098b3 nodeName:}" failed. No retries permitted until 2026-07-29 21:59:09.642484369 +0000 UTC m=+3301.523758484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0020d846-4a8a-487b-9fe6-d03832e098b3-maas-api-tls") pod "maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf" (UID: "0020d846-4a8a-487b-9fe6-d03832e098b3") : secret "maas-api-e2e-ipp-b-50d50c-serving-cert" not found
Jul 29 21:59:09.142948 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:09.142930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0020d846-4a8a-487b-9fe6-d03832e098b3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf\" (UID: \"0020d846-4a8a-487b-9fe6-d03832e098b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:09.151463 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:09.151440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pqsr\" (UniqueName: \"kubernetes.io/projected/0020d846-4a8a-487b-9fe6-d03832e098b3-kube-api-access-7pqsr\") pod \"maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf\" (UID: \"0020d846-4a8a-487b-9fe6-d03832e098b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:09.646656 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:09.646624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0020d846-4a8a-487b-9fe6-d03832e098b3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf\" (UID: \"0020d846-4a8a-487b-9fe6-d03832e098b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:09.648980 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:09.648953    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0020d846-4a8a-487b-9fe6-d03832e098b3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf\" (UID: \"0020d846-4a8a-487b-9fe6-d03832e098b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:09.851268 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:09.851236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:10.188097 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:10.188022    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"]
Jul 29 21:59:10.191735 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:59:10.191704    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0020d846_4a8a_487b_9fe6_d03832e098b3.slice/crio-0859aeb1059813256d9bdd54f4aa47bb50b2e47cf105089968d9aff23003bef1 WatchSource:0}: Error finding container 0859aeb1059813256d9bdd54f4aa47bb50b2e47cf105089968d9aff23003bef1: Status 404 returned error can't find the container with id 0859aeb1059813256d9bdd54f4aa47bb50b2e47cf105089968d9aff23003bef1
Jul 29 21:59:10.692383 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:10.692358    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf" event={"ID":"0020d846-4a8a-487b-9fe6-d03832e098b3","Type":"ContainerStarted","Data":"0859aeb1059813256d9bdd54f4aa47bb50b2e47cf105089968d9aff23003bef1"}
Jul 29 21:59:11.698998 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:11.698961    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf" event={"ID":"0020d846-4a8a-487b-9fe6-d03832e098b3","Type":"ContainerStarted","Data":"a29d9c91b9dec90e89e440628910ea7fc0b4bedc302c9bcd43540955763f7aa7"}
Jul 29 21:59:11.699606 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:11.699083    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:11.717916 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:11.717872    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf" podStartSLOduration=3.280325765 podStartE2EDuration="3.717858145s" totalImagesPullingTime="437.53238ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:59:11.716142382 +0000 UTC m=+3303.597416560" watchObservedRunningTime="2026-07-29 21:59:11.717858145 +0000 UTC m=+3303.599132281"
Jul 29 21:59:17.709281 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:17.709248    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 21:59:52.995499 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:52.995464    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"]
Jul 29 21:59:52.998816 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:52.998793    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.001874 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.001834    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-a0dedf\"/\"default-dockercfg-bppkz\""
Jul 29 21:59:53.003267 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.003245    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-a0dedf\"/\"kube-root-ca.crt\""
Jul 29 21:59:53.003379 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.003280    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-a0dedf\"/\"ipp-route-a0dedf-kserve-self-signed-certs\""
Jul 29 21:59:53.003379 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.003320    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-a0dedf\"/\"openshift-service-ca.crt\""
Jul 29 21:59:53.009097 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.009074    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"]
Jul 29 21:59:53.163903 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.163875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-dshm\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.164172 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.164125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-tmp-dir\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.164317 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.164302    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-model-cache\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.164443 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.164416    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-tls-certs\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.164573 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.164454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-home\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.164573 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.164514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdd8x\" (UniqueName: \"kubernetes.io/projected/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-kube-api-access-jdd8x\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.265745 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.265678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-dshm\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.265745 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.265707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-tmp-dir\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.265745 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.265726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-model-cache\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.265745 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.265742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-tls-certs\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.266053 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.265761    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-home\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.266053 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.265805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdd8x\" (UniqueName: \"kubernetes.io/projected/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-kube-api-access-jdd8x\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.266177 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.266154    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-model-cache\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.266258 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.266200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-tmp-dir\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.266258 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.266233    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-home\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.268012 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.267993    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-dshm\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.268274 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.268258    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-tls-certs\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.273694 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.273674    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdd8x\" (UniqueName: \"kubernetes.io/projected/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-kube-api-access-jdd8x\") pod \"ipp-route-a0dedf-kserve-666b44dd8c-hppd4\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") " pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.308587 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.308570    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 21:59:53.431127 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.431101    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"]
Jul 29 21:59:53.432642 ip-10-0-138-99 kubenswrapper[2580]: W0729 21:59:53.432614    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcdd50c6e_4658_4ca7_b6e8_1b024dce1f89.slice/crio-e3930838d8865e21bdf8f27d2ddd07b482f6a85a3302bdb32158914a535412d9 WatchSource:0}: Error finding container e3930838d8865e21bdf8f27d2ddd07b482f6a85a3302bdb32158914a535412d9: Status 404 returned error can't find the container with id e3930838d8865e21bdf8f27d2ddd07b482f6a85a3302bdb32158914a535412d9
Jul 29 21:59:53.898290 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.898243    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4" event={"ID":"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89","Type":"ContainerStarted","Data":"f5e6e943d76db54f0a263db6b7ef369e84097eb006ecb5fd9c363e1f1ff737b1"}
Jul 29 21:59:53.898538 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.898516    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4" event={"ID":"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89","Type":"ContainerStarted","Data":"e3930838d8865e21bdf8f27d2ddd07b482f6a85a3302bdb32158914a535412d9"}
Jul 29 21:59:53.919910 ip-10-0-138-99 kubenswrapper[2580]: I0729 21:59:53.919864    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4" podStartSLOduration=1.919850625 podStartE2EDuration="1.919850625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:59:53.917560839 +0000 UTC m=+3345.798834977" watchObservedRunningTime="2026-07-29 21:59:53.919850625 +0000 UTC m=+3345.801124753"
Jul 29 22:00:00.144593 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:00.144543    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7"]
Jul 29 22:00:00.148635 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:00.148613    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a0dedf-29756040-blg2p"]
Jul 29 22:00:00.148760 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:00.148741    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7"
Jul 29 22:00:00.151802 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:00.151769    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9wjvq\""
Jul 29 22:00:00.153186 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:00.153162    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a0dedf-29756040-blg2p"
Jul 29 22:00:00.158015 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:00.157984    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7"]
Jul 29 22:00:00.159764 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:00.159738    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a0dedf-29756040-blg2p"]
Jul 29 22:00:00.162922 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:00.162903    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7"
Jul 29 22:00:00.171318 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:00.171296    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a0dedf-29756040-blg2p"
Jul 29 22:00:00.308097 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:00.308073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7"]
Jul 29 22:00:00.311490 ip-10-0-138-99 kubenswrapper[2580]: W0729 22:00:00.311461    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded03e702_032f_411c_87b3_4c336bf97424.slice/crio-7fb51a0268f1e3a74b38ff86f07c2c50c67f1f72b88f303313ccf07a7e0748ed WatchSource:0}: Error finding container 7fb51a0268f1e3a74b38ff86f07c2c50c67f1f72b88f303313ccf07a7e0748ed: Status 404 returned error can't find the container with id 7fb51a0268f1e3a74b38ff86f07c2c50c67f1f72b88f303313ccf07a7e0748ed
Jul 29 22:00:00.328342 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:00.328318    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a0dedf-29756040-blg2p"]
Jul 29 22:00:00.331433 ip-10-0-138-99 kubenswrapper[2580]: W0729 22:00:00.331408    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d6494f_12dd_4420_baf9_4e1559c7a132.slice/crio-cc839fc98041dce15c80bc664f8b23cfc109ec9461088ff152af4139dd835183 WatchSource:0}: Error finding container cc839fc98041dce15c80bc664f8b23cfc109ec9461088ff152af4139dd835183: Status 404 returned error can't find the container with id cc839fc98041dce15c80bc664f8b23cfc109ec9461088ff152af4139dd835183
Jul 29 22:00:00.932768 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:00.932733    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7" event={"ID":"ed03e702-032f-411c-87b3-4c336bf97424","Type":"ContainerStarted","Data":"7fb51a0268f1e3a74b38ff86f07c2c50c67f1f72b88f303313ccf07a7e0748ed"}
Jul 29 22:00:00.934340 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:00.934305    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a0dedf-29756040-blg2p" event={"ID":"c3d6494f-12dd-4420-baf9-4e1559c7a132","Type":"ContainerStarted","Data":"cc839fc98041dce15c80bc664f8b23cfc109ec9461088ff152af4139dd835183"}
Jul 29 22:00:02.946880 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:02.946851    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7" event={"ID":"ed03e702-032f-411c-87b3-4c336bf97424","Type":"ContainerStarted","Data":"181536cfa49067bb739ceb67f4519252b685fa4933084181267169e9f9acff27"}
Jul 29 22:00:02.961858 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:02.961800    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7" podStartSLOduration=0.539938152 podStartE2EDuration="2.961786647s" totalImagesPullingTime="2.421848495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:00:02.961210486 +0000 UTC m=+3354.842484624" watchObservedRunningTime="2026-07-29 22:00:02.961786647 +0000 UTC m=+3354.843060842"
Jul 29 22:00:03.309706 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:03.309639    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 22:00:03.309706 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:03.309678    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 22:00:03.321486 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:03.321463    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 22:00:03.954093 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:03.954053    2580 generic.go:356] "Generic (PLEG): container finished" podID="ed03e702-032f-411c-87b3-4c336bf97424" containerID="181536cfa49067bb739ceb67f4519252b685fa4933084181267169e9f9acff27" exitCode=0
Jul 29 22:00:03.954499 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:03.954137    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7" event={"ID":"ed03e702-032f-411c-87b3-4c336bf97424","Type":"ContainerDied","Data":"181536cfa49067bb739ceb67f4519252b685fa4933084181267169e9f9acff27"}
Jul 29 22:00:03.956142 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:03.956109    2580 generic.go:356] "Generic (PLEG): container finished" podID="c3d6494f-12dd-4420-baf9-4e1559c7a132" containerID="67ccdfbb9f848d168483cabf556c9a200606eb64334e4c20a34c3279105aa468" exitCode=0
Jul 29 22:00:03.956483 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:03.956162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a0dedf-29756040-blg2p" event={"ID":"c3d6494f-12dd-4420-baf9-4e1559c7a132","Type":"ContainerDied","Data":"67ccdfbb9f848d168483cabf556c9a200606eb64334e4c20a34c3279105aa468"}
Jul 29 22:00:03.967329 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:03.967309    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 22:00:05.093416 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:05.093395    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7"
Jul 29 22:00:05.123089 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:05.123068    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a0dedf-29756040-blg2p"
Jul 29 22:00:05.967580 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:05.967538    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7"
Jul 29 22:00:05.967781 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:05.967582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7" event={"ID":"ed03e702-032f-411c-87b3-4c336bf97424","Type":"ContainerDied","Data":"7fb51a0268f1e3a74b38ff86f07c2c50c67f1f72b88f303313ccf07a7e0748ed"}
Jul 29 22:00:05.967781 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:05.967633    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fb51a0268f1e3a74b38ff86f07c2c50c67f1f72b88f303313ccf07a7e0748ed"
Jul 29 22:00:05.969107 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:05.969084    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a0dedf-29756040-blg2p"
Jul 29 22:00:05.969249 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:05.969084    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a0dedf-29756040-blg2p" event={"ID":"c3d6494f-12dd-4420-baf9-4e1559c7a132","Type":"ContainerDied","Data":"cc839fc98041dce15c80bc664f8b23cfc109ec9461088ff152af4139dd835183"}
Jul 29 22:00:05.969249 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:05.969200    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc839fc98041dce15c80bc664f8b23cfc109ec9461088ff152af4139dd835183"
Jul 29 22:00:40.470639 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.470605    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"]
Jul 29 22:00:40.471131 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.470962    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d6494f-12dd-4420-baf9-4e1559c7a132" containerName="cleanup"
Jul 29 22:00:40.471131 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.470976    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d6494f-12dd-4420-baf9-4e1559c7a132" containerName="cleanup"
Jul 29 22:00:40.471131 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.470990    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed03e702-032f-411c-87b3-4c336bf97424" containerName="cleanup"
Jul 29 22:00:40.471131 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.470995    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed03e702-032f-411c-87b3-4c336bf97424" containerName="cleanup"
Jul 29 22:00:40.471131 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.471067    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3d6494f-12dd-4420-baf9-4e1559c7a132" containerName="cleanup"
Jul 29 22:00:40.471131 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.471076    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed03e702-032f-411c-87b3-4c336bf97424" containerName="cleanup"
Jul 29 22:00:40.474089 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.474070    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:00:40.477397 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.477378    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-215280-serving-cert\""
Jul 29 22:00:40.486089 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.486066    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"]
Jul 29 22:00:40.615461 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.615430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2496c190-6136-4eb2-82c6-339551f35cf7-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl\" (UID: \"2496c190-6136-4eb2-82c6-339551f35cf7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:00:40.615609 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.615468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2496c190-6136-4eb2-82c6-339551f35cf7-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl\" (UID: \"2496c190-6136-4eb2-82c6-339551f35cf7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:00:40.615609 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.615501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9w44\" (UniqueName: \"kubernetes.io/projected/2496c190-6136-4eb2-82c6-339551f35cf7-kube-api-access-g9w44\") pod \"maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl\" (UID: \"2496c190-6136-4eb2-82c6-339551f35cf7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:00:40.716380 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.716352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2496c190-6136-4eb2-82c6-339551f35cf7-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl\" (UID: \"2496c190-6136-4eb2-82c6-339551f35cf7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:00:40.716507 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.716387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9w44\" (UniqueName: \"kubernetes.io/projected/2496c190-6136-4eb2-82c6-339551f35cf7-kube-api-access-g9w44\") pod \"maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl\" (UID: \"2496c190-6136-4eb2-82c6-339551f35cf7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:00:40.716507 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.716440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2496c190-6136-4eb2-82c6-339551f35cf7-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl\" (UID: \"2496c190-6136-4eb2-82c6-339551f35cf7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:00:40.717054 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.717034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2496c190-6136-4eb2-82c6-339551f35cf7-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl\" (UID: \"2496c190-6136-4eb2-82c6-339551f35cf7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:00:40.718888 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.718868    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2496c190-6136-4eb2-82c6-339551f35cf7-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl\" (UID: \"2496c190-6136-4eb2-82c6-339551f35cf7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:00:40.724683 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.724637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9w44\" (UniqueName: \"kubernetes.io/projected/2496c190-6136-4eb2-82c6-339551f35cf7-kube-api-access-g9w44\") pod \"maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl\" (UID: \"2496c190-6136-4eb2-82c6-339551f35cf7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:00:40.783511 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.783489    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:00:40.913079 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:40.913050    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"]
Jul 29 22:00:40.914642 ip-10-0-138-99 kubenswrapper[2580]: W0729 22:00:40.914618    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2496c190_6136_4eb2_82c6_339551f35cf7.slice/crio-42e7671df2110fe0063ee67449ce5fd1e752cc8a66ecee2cb88a973e19e00be1 WatchSource:0}: Error finding container 42e7671df2110fe0063ee67449ce5fd1e752cc8a66ecee2cb88a973e19e00be1: Status 404 returned error can't find the container with id 42e7671df2110fe0063ee67449ce5fd1e752cc8a66ecee2cb88a973e19e00be1
Jul 29 22:00:41.145506 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:41.145475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl" event={"ID":"2496c190-6136-4eb2-82c6-339551f35cf7","Type":"ContainerStarted","Data":"42e7671df2110fe0063ee67449ce5fd1e752cc8a66ecee2cb88a973e19e00be1"}
Jul 29 22:00:42.153514 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:42.153474    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl" event={"ID":"2496c190-6136-4eb2-82c6-339551f35cf7","Type":"ContainerStarted","Data":"1c069c8ed1f4704088cde0160ce8878b30989b5163dc040ee19bc33303e1060c"}
Jul 29 22:00:42.153967 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:42.153595    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:00:42.173851 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:42.173795    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl" podStartSLOduration=1.679829993 podStartE2EDuration="2.173777711s" totalImagesPullingTime="493.947718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:00:42.170903405 +0000 UTC m=+3394.052177562" watchObservedRunningTime="2026-07-29 22:00:42.173777711 +0000 UTC m=+3394.055051848"
Jul 29 22:00:48.165760 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:00:48.165718    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:01:05.199264 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:05.199186    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6"]
Jul 29 22:01:05.202619 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:05.202595    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6"
Jul 29 22:01:05.205422 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:05.205397    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9wjvq\""
Jul 29 22:01:05.211175 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:05.210680    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6"]
Jul 29 22:01:05.314548 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:05.314523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e85f4457-6b68-4d0d-a31d-a526a9394b8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6\" (UID: \"e85f4457-6b68-4d0d-a31d-a526a9394b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6"
Jul 29 22:01:05.415385 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:05.415364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e85f4457-6b68-4d0d-a31d-a526a9394b8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6\" (UID: \"e85f4457-6b68-4d0d-a31d-a526a9394b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6"
Jul 29 22:01:05.415979 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:05.415959    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e85f4457-6b68-4d0d-a31d-a526a9394b8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6\" (UID: \"e85f4457-6b68-4d0d-a31d-a526a9394b8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6"
Jul 29 22:01:05.515699 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:05.515639    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6"
Jul 29 22:01:05.649239 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:05.649209    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6"]
Jul 29 22:01:05.654600 ip-10-0-138-99 kubenswrapper[2580]: W0729 22:01:05.654571    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85f4457_6b68_4d0d_a31d_a526a9394b8d.slice/crio-79ae89b98cc225cca20192fae9dc2f087e66ec221b4dd7b7d92c6e795b862683 WatchSource:0}: Error finding container 79ae89b98cc225cca20192fae9dc2f087e66ec221b4dd7b7d92c6e795b862683: Status 404 returned error can't find the container with id 79ae89b98cc225cca20192fae9dc2f087e66ec221b4dd7b7d92c6e795b862683
Jul 29 22:01:06.272765 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:06.272726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6" event={"ID":"e85f4457-6b68-4d0d-a31d-a526a9394b8d","Type":"ContainerStarted","Data":"660a76c284ee91989087d263c50ee30234dfdd0da4759f9e2fd3f1727c067bec"}
Jul 29 22:01:06.273110 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:06.272788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6" event={"ID":"e85f4457-6b68-4d0d-a31d-a526a9394b8d","Type":"ContainerStarted","Data":"79ae89b98cc225cca20192fae9dc2f087e66ec221b4dd7b7d92c6e795b862683"}
Jul 29 22:01:06.290472 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:06.290427    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6" podStartSLOduration=1.290412881 podStartE2EDuration="1.290412881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:01:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:01:06.290086276 +0000 UTC m=+3418.171360423" watchObservedRunningTime="2026-07-29 22:01:06.290412881 +0000 UTC m=+3418.171687017"
Jul 29 22:01:07.279073 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:07.279034    2580 generic.go:356] "Generic (PLEG): container finished" podID="e85f4457-6b68-4d0d-a31d-a526a9394b8d" containerID="660a76c284ee91989087d263c50ee30234dfdd0da4759f9e2fd3f1727c067bec" exitCode=0
Jul 29 22:01:07.279502 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:07.279131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6" event={"ID":"e85f4457-6b68-4d0d-a31d-a526a9394b8d","Type":"ContainerDied","Data":"660a76c284ee91989087d263c50ee30234dfdd0da4759f9e2fd3f1727c067bec"}
Jul 29 22:01:08.407047 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:08.407025    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6"
Jul 29 22:01:08.540075 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:08.540004    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e85f4457-6b68-4d0d-a31d-a526a9394b8d-maas-api-service-ca\") pod \"e85f4457-6b68-4d0d-a31d-a526a9394b8d\" (UID: \"e85f4457-6b68-4d0d-a31d-a526a9394b8d\") "
Jul 29 22:01:08.540310 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:08.540288    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e85f4457-6b68-4d0d-a31d-a526a9394b8d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e85f4457-6b68-4d0d-a31d-a526a9394b8d" (UID: "e85f4457-6b68-4d0d-a31d-a526a9394b8d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:01:08.641404 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:08.641377    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e85f4457-6b68-4d0d-a31d-a526a9394b8d-maas-api-service-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:09.290336 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:09.290307    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6"
Jul 29 22:01:09.290523 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:09.290310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6" event={"ID":"e85f4457-6b68-4d0d-a31d-a526a9394b8d","Type":"ContainerDied","Data":"79ae89b98cc225cca20192fae9dc2f087e66ec221b4dd7b7d92c6e795b862683"}
Jul 29 22:01:09.290523 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:09.290447    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79ae89b98cc225cca20192fae9dc2f087e66ec221b4dd7b7d92c6e795b862683"
Jul 29 22:01:10.260526 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.260490    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"]
Jul 29 22:01:10.260998 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.260767    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl" podUID="2496c190-6136-4eb2-82c6-339551f35cf7" containerName="maas-api" containerID="cri-o://1c069c8ed1f4704088cde0160ce8878b30989b5163dc040ee19bc33303e1060c" gracePeriod=30
Jul 29 22:01:10.505860 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.505815    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:01:10.657006 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.656978    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9w44\" (UniqueName: \"kubernetes.io/projected/2496c190-6136-4eb2-82c6-339551f35cf7-kube-api-access-g9w44\") pod \"2496c190-6136-4eb2-82c6-339551f35cf7\" (UID: \"2496c190-6136-4eb2-82c6-339551f35cf7\") "
Jul 29 22:01:10.657189 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.657060    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2496c190-6136-4eb2-82c6-339551f35cf7-maas-api-tls\") pod \"2496c190-6136-4eb2-82c6-339551f35cf7\" (UID: \"2496c190-6136-4eb2-82c6-339551f35cf7\") "
Jul 29 22:01:10.657189 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.657103    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2496c190-6136-4eb2-82c6-339551f35cf7-openshift-service-ca\") pod \"2496c190-6136-4eb2-82c6-339551f35cf7\" (UID: \"2496c190-6136-4eb2-82c6-339551f35cf7\") "
Jul 29 22:01:10.657492 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.657469    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2496c190-6136-4eb2-82c6-339551f35cf7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2496c190-6136-4eb2-82c6-339551f35cf7" (UID: "2496c190-6136-4eb2-82c6-339551f35cf7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:01:10.659265 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.659236    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2496c190-6136-4eb2-82c6-339551f35cf7-kube-api-access-g9w44" (OuterVolumeSpecName: "kube-api-access-g9w44") pod "2496c190-6136-4eb2-82c6-339551f35cf7" (UID: "2496c190-6136-4eb2-82c6-339551f35cf7"). InnerVolumeSpecName "kube-api-access-g9w44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:01:10.659360 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.659242    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2496c190-6136-4eb2-82c6-339551f35cf7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2496c190-6136-4eb2-82c6-339551f35cf7" (UID: "2496c190-6136-4eb2-82c6-339551f35cf7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:01:10.674442 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.674396    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6"]
Jul 29 22:01:10.676297 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.676273    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-215280-4kxp6"]
Jul 29 22:01:10.716164 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.716126    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e85f4457-6b68-4d0d-a31d-a526a9394b8d" path="/var/lib/kubelet/pods/e85f4457-6b68-4d0d-a31d-a526a9394b8d/volumes"
Jul 29 22:01:10.757913 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.757893    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2496c190-6136-4eb2-82c6-339551f35cf7-openshift-service-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:10.757913 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.757911    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9w44\" (UniqueName: \"kubernetes.io/projected/2496c190-6136-4eb2-82c6-339551f35cf7-kube-api-access-g9w44\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:10.758027 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:10.757920    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2496c190-6136-4eb2-82c6-339551f35cf7-maas-api-tls\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:11.305191 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:11.305156    2580 generic.go:356] "Generic (PLEG): container finished" podID="2496c190-6136-4eb2-82c6-339551f35cf7" containerID="1c069c8ed1f4704088cde0160ce8878b30989b5163dc040ee19bc33303e1060c" exitCode=0
Jul 29 22:01:11.305584 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:11.305209    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"
Jul 29 22:01:11.305584 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:11.305240    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl" event={"ID":"2496c190-6136-4eb2-82c6-339551f35cf7","Type":"ContainerDied","Data":"1c069c8ed1f4704088cde0160ce8878b30989b5163dc040ee19bc33303e1060c"}
Jul 29 22:01:11.305584 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:11.305303    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl" event={"ID":"2496c190-6136-4eb2-82c6-339551f35cf7","Type":"ContainerDied","Data":"42e7671df2110fe0063ee67449ce5fd1e752cc8a66ecee2cb88a973e19e00be1"}
Jul 29 22:01:11.305584 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:11.305328    2580 scope.go:117] "RemoveContainer" containerID="1c069c8ed1f4704088cde0160ce8878b30989b5163dc040ee19bc33303e1060c"
Jul 29 22:01:11.315366 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:11.315344    2580 scope.go:117] "RemoveContainer" containerID="1c069c8ed1f4704088cde0160ce8878b30989b5163dc040ee19bc33303e1060c"
Jul 29 22:01:11.315703 ip-10-0-138-99 kubenswrapper[2580]: E0729 22:01:11.315681    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c069c8ed1f4704088cde0160ce8878b30989b5163dc040ee19bc33303e1060c\": container with ID starting with 1c069c8ed1f4704088cde0160ce8878b30989b5163dc040ee19bc33303e1060c not found: ID does not exist" containerID="1c069c8ed1f4704088cde0160ce8878b30989b5163dc040ee19bc33303e1060c"
Jul 29 22:01:11.315757 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:11.315713    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c069c8ed1f4704088cde0160ce8878b30989b5163dc040ee19bc33303e1060c"} err="failed to get container status \"1c069c8ed1f4704088cde0160ce8878b30989b5163dc040ee19bc33303e1060c\": rpc error: code = NotFound desc = could not find container \"1c069c8ed1f4704088cde0160ce8878b30989b5163dc040ee19bc33303e1060c\": container with ID starting with 1c069c8ed1f4704088cde0160ce8878b30989b5163dc040ee19bc33303e1060c not found: ID does not exist"
Jul 29 22:01:11.322130 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:11.322104    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"]
Jul 29 22:01:11.325342 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:11.325317    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-215280-575b5f8b87-shjzl"]
Jul 29 22:01:12.715196 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:12.715160    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2496c190-6136-4eb2-82c6-339551f35cf7" path="/var/lib/kubelet/pods/2496c190-6136-4eb2-82c6-339551f35cf7/volumes"
Jul 29 22:01:21.010004 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.009961    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz"]
Jul 29 22:01:21.010403 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.010351    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e85f4457-6b68-4d0d-a31d-a526a9394b8d" containerName="revoke-keys"
Jul 29 22:01:21.010403 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.010365    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e85f4457-6b68-4d0d-a31d-a526a9394b8d" containerName="revoke-keys"
Jul 29 22:01:21.010403 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.010379    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2496c190-6136-4eb2-82c6-339551f35cf7" containerName="maas-api"
Jul 29 22:01:21.010403 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.010385    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2496c190-6136-4eb2-82c6-339551f35cf7" containerName="maas-api"
Jul 29 22:01:21.010553 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.010452    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2496c190-6136-4eb2-82c6-339551f35cf7" containerName="maas-api"
Jul 29 22:01:21.010553 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.010463    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e85f4457-6b68-4d0d-a31d-a526a9394b8d" containerName="revoke-keys"
Jul 29 22:01:21.013483 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.013461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz"
Jul 29 22:01:21.016388 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.016354    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9wjvq\""
Jul 29 22:01:21.023527 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.023497    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz"]
Jul 29 22:01:21.030470 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.030436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30364cd-d45e-4917-a926-c5d55df0dbd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz\" (UID: \"b30364cd-d45e-4917-a926-c5d55df0dbd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz"
Jul 29 22:01:21.131847 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.131805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30364cd-d45e-4917-a926-c5d55df0dbd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz\" (UID: \"b30364cd-d45e-4917-a926-c5d55df0dbd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz"
Jul 29 22:01:21.132430 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.132407    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30364cd-d45e-4917-a926-c5d55df0dbd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz\" (UID: \"b30364cd-d45e-4917-a926-c5d55df0dbd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz"
Jul 29 22:01:21.326488 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.326421    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz"
Jul 29 22:01:21.455767 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:21.455742    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz"]
Jul 29 22:01:21.457815 ip-10-0-138-99 kubenswrapper[2580]: W0729 22:01:21.457777    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb30364cd_d45e_4917_a926_c5d55df0dbd9.slice/crio-b2e22be9cbc8411b1e52057534e976b063f30f72aaad988c343d2fda5deab5ea WatchSource:0}: Error finding container b2e22be9cbc8411b1e52057534e976b063f30f72aaad988c343d2fda5deab5ea: Status 404 returned error can't find the container with id b2e22be9cbc8411b1e52057534e976b063f30f72aaad988c343d2fda5deab5ea
Jul 29 22:01:22.362045 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:22.362008    2580 generic.go:356] "Generic (PLEG): container finished" podID="b30364cd-d45e-4917-a926-c5d55df0dbd9" containerID="8937fac1506eb831cd102c7febb463b2537e6a723038cfd9c0ee730b96554f2d" exitCode=0
Jul 29 22:01:22.362397 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:22.362063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz" event={"ID":"b30364cd-d45e-4917-a926-c5d55df0dbd9","Type":"ContainerDied","Data":"8937fac1506eb831cd102c7febb463b2537e6a723038cfd9c0ee730b96554f2d"}
Jul 29 22:01:22.362397 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:22.362160    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz" event={"ID":"b30364cd-d45e-4917-a926-c5d55df0dbd9","Type":"ContainerStarted","Data":"b2e22be9cbc8411b1e52057534e976b063f30f72aaad988c343d2fda5deab5ea"}
Jul 29 22:01:23.490601 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:23.490576    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz"
Jul 29 22:01:23.551171 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:23.551147    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30364cd-d45e-4917-a926-c5d55df0dbd9-maas-api-service-ca\") pod \"b30364cd-d45e-4917-a926-c5d55df0dbd9\" (UID: \"b30364cd-d45e-4917-a926-c5d55df0dbd9\") "
Jul 29 22:01:23.551463 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:23.551443    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b30364cd-d45e-4917-a926-c5d55df0dbd9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b30364cd-d45e-4917-a926-c5d55df0dbd9" (UID: "b30364cd-d45e-4917-a926-c5d55df0dbd9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:01:23.652517 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:23.652492    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30364cd-d45e-4917-a926-c5d55df0dbd9-maas-api-service-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:24.372699 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:24.372672    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz"
Jul 29 22:01:24.372879 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:24.372706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz" event={"ID":"b30364cd-d45e-4917-a926-c5d55df0dbd9","Type":"ContainerDied","Data":"b2e22be9cbc8411b1e52057534e976b063f30f72aaad988c343d2fda5deab5ea"}
Jul 29 22:01:24.372879 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:24.372759    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2e22be9cbc8411b1e52057534e976b063f30f72aaad988c343d2fda5deab5ea"
Jul 29 22:01:36.451687 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.451653    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"]
Jul 29 22:01:36.452154 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.451947    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv" podUID="920a1d5a-b13c-48c9-819a-040a450aae3b" containerName="maas-api" containerID="cri-o://539194e6da06fab5194eaf59670a276f9002c6419d9ca0d9120e8d00f9bba120" gracePeriod=30
Jul 29 22:01:36.470447 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.470420    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a0dedf-29756040-blg2p"]
Jul 29 22:01:36.476446 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.476423    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a0dedf-29756040-blg2p"]
Jul 29 22:01:36.648958 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.648929    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz"]
Jul 29 22:01:36.654181 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.654153    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a0dedf-l2sbz"]
Jul 29 22:01:36.710721 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.710667    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 22:01:36.715112 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.715090    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b30364cd-d45e-4917-a926-c5d55df0dbd9" path="/var/lib/kubelet/pods/b30364cd-d45e-4917-a926-c5d55df0dbd9/volumes"
Jul 29 22:01:36.715427 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.715412    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3d6494f-12dd-4420-baf9-4e1559c7a132" path="/var/lib/kubelet/pods/c3d6494f-12dd-4420-baf9-4e1559c7a132/volumes"
Jul 29 22:01:36.747636 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.747611    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/920a1d5a-b13c-48c9-819a-040a450aae3b-openshift-service-ca\") pod \"920a1d5a-b13c-48c9-819a-040a450aae3b\" (UID: \"920a1d5a-b13c-48c9-819a-040a450aae3b\") "
Jul 29 22:01:36.747759 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.747659    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dbv7\" (UniqueName: \"kubernetes.io/projected/920a1d5a-b13c-48c9-819a-040a450aae3b-kube-api-access-7dbv7\") pod \"920a1d5a-b13c-48c9-819a-040a450aae3b\" (UID: \"920a1d5a-b13c-48c9-819a-040a450aae3b\") "
Jul 29 22:01:36.747759 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.747686    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/920a1d5a-b13c-48c9-819a-040a450aae3b-maas-api-tls\") pod \"920a1d5a-b13c-48c9-819a-040a450aae3b\" (UID: \"920a1d5a-b13c-48c9-819a-040a450aae3b\") "
Jul 29 22:01:36.748066 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.748028    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/920a1d5a-b13c-48c9-819a-040a450aae3b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "920a1d5a-b13c-48c9-819a-040a450aae3b" (UID: "920a1d5a-b13c-48c9-819a-040a450aae3b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:01:36.749760 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.749738    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/920a1d5a-b13c-48c9-819a-040a450aae3b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "920a1d5a-b13c-48c9-819a-040a450aae3b" (UID: "920a1d5a-b13c-48c9-819a-040a450aae3b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:01:36.749857 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.749780    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/920a1d5a-b13c-48c9-819a-040a450aae3b-kube-api-access-7dbv7" (OuterVolumeSpecName: "kube-api-access-7dbv7") pod "920a1d5a-b13c-48c9-819a-040a450aae3b" (UID: "920a1d5a-b13c-48c9-819a-040a450aae3b"). InnerVolumeSpecName "kube-api-access-7dbv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:01:36.849036 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.849013    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/920a1d5a-b13c-48c9-819a-040a450aae3b-openshift-service-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:36.849036 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.849035    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dbv7\" (UniqueName: \"kubernetes.io/projected/920a1d5a-b13c-48c9-819a-040a450aae3b-kube-api-access-7dbv7\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:36.849195 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:36.849045    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/920a1d5a-b13c-48c9-819a-040a450aae3b-maas-api-tls\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:37.438231 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:37.438191    2580 generic.go:356] "Generic (PLEG): container finished" podID="920a1d5a-b13c-48c9-819a-040a450aae3b" containerID="539194e6da06fab5194eaf59670a276f9002c6419d9ca0d9120e8d00f9bba120" exitCode=0
Jul 29 22:01:37.438415 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:37.438238    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"
Jul 29 22:01:37.438415 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:37.438279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv" event={"ID":"920a1d5a-b13c-48c9-819a-040a450aae3b","Type":"ContainerDied","Data":"539194e6da06fab5194eaf59670a276f9002c6419d9ca0d9120e8d00f9bba120"}
Jul 29 22:01:37.438415 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:37.438330    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv" event={"ID":"920a1d5a-b13c-48c9-819a-040a450aae3b","Type":"ContainerDied","Data":"3cf58e388f5fdfedada59538219c4b5e92fc2cb5d0a239d0a30c48aa7f5c8ff0"}
Jul 29 22:01:37.438415 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:37.438354    2580 scope.go:117] "RemoveContainer" containerID="539194e6da06fab5194eaf59670a276f9002c6419d9ca0d9120e8d00f9bba120"
Jul 29 22:01:37.447097 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:37.447079    2580 scope.go:117] "RemoveContainer" containerID="539194e6da06fab5194eaf59670a276f9002c6419d9ca0d9120e8d00f9bba120"
Jul 29 22:01:37.447332 ip-10-0-138-99 kubenswrapper[2580]: E0729 22:01:37.447315    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"539194e6da06fab5194eaf59670a276f9002c6419d9ca0d9120e8d00f9bba120\": container with ID starting with 539194e6da06fab5194eaf59670a276f9002c6419d9ca0d9120e8d00f9bba120 not found: ID does not exist" containerID="539194e6da06fab5194eaf59670a276f9002c6419d9ca0d9120e8d00f9bba120"
Jul 29 22:01:37.447376 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:37.447342    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"539194e6da06fab5194eaf59670a276f9002c6419d9ca0d9120e8d00f9bba120"} err="failed to get container status \"539194e6da06fab5194eaf59670a276f9002c6419d9ca0d9120e8d00f9bba120\": rpc error: code = NotFound desc = could not find container \"539194e6da06fab5194eaf59670a276f9002c6419d9ca0d9120e8d00f9bba120\": container with ID starting with 539194e6da06fab5194eaf59670a276f9002c6419d9ca0d9120e8d00f9bba120 not found: ID does not exist"
Jul 29 22:01:37.459822 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:37.459800    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"]
Jul 29 22:01:37.466345 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:37.466326    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a0dedf-795f5f8856-8x2wv"]
Jul 29 22:01:38.715008 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:38.714969    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="920a1d5a-b13c-48c9-819a-040a450aae3b" path="/var/lib/kubelet/pods/920a1d5a-b13c-48c9-819a-040a450aae3b/volumes"
Jul 29 22:01:42.007685 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:42.007645    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"]
Jul 29 22:01:57.027343 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.027306    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4" podUID="cdd50c6e-4658-4ca7-b6e8-1b024dce1f89" containerName="main" containerID="cri-o://f5e6e943d76db54f0a263db6b7ef369e84097eb006ecb5fd9c363e1f1ff737b1" gracePeriod=45
Jul 29 22:01:57.265609 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.265584    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 22:01:57.400646 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.400621    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-home\") pod \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") "
Jul 29 22:01:57.400799 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.400667    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-model-cache\") pod \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") "
Jul 29 22:01:57.400799 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.400690    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdd8x\" (UniqueName: \"kubernetes.io/projected/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-kube-api-access-jdd8x\") pod \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") "
Jul 29 22:01:57.400799 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.400726    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-tmp-dir\") pod \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") "
Jul 29 22:01:57.400799 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.400742    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-tls-certs\") pod \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") "
Jul 29 22:01:57.400799 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.400759    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-dshm\") pod \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\" (UID: \"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89\") "
Jul 29 22:01:57.401096 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.400942    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-home" (OuterVolumeSpecName: "home") pod "cdd50c6e-4658-4ca7-b6e8-1b024dce1f89" (UID: "cdd50c6e-4658-4ca7-b6e8-1b024dce1f89"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:01:57.401096 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.400978    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-model-cache" (OuterVolumeSpecName: "model-cache") pod "cdd50c6e-4658-4ca7-b6e8-1b024dce1f89" (UID: "cdd50c6e-4658-4ca7-b6e8-1b024dce1f89"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:01:57.401096 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.401045    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cdd50c6e-4658-4ca7-b6e8-1b024dce1f89" (UID: "cdd50c6e-4658-4ca7-b6e8-1b024dce1f89"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:01:57.402901 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.402868    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-dshm" (OuterVolumeSpecName: "dshm") pod "cdd50c6e-4658-4ca7-b6e8-1b024dce1f89" (UID: "cdd50c6e-4658-4ca7-b6e8-1b024dce1f89"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:01:57.402993 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.402968    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-kube-api-access-jdd8x" (OuterVolumeSpecName: "kube-api-access-jdd8x") pod "cdd50c6e-4658-4ca7-b6e8-1b024dce1f89" (UID: "cdd50c6e-4658-4ca7-b6e8-1b024dce1f89"). InnerVolumeSpecName "kube-api-access-jdd8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:01:57.403048 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.402970    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cdd50c6e-4658-4ca7-b6e8-1b024dce1f89" (UID: "cdd50c6e-4658-4ca7-b6e8-1b024dce1f89"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:01:57.501416 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.501390    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-dshm\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:57.501416 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.501412    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-home\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:57.501550 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.501420    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-model-cache\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:57.501550 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.501429    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdd8x\" (UniqueName: \"kubernetes.io/projected/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-kube-api-access-jdd8x\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:57.501550 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.501438    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-tmp-dir\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:57.501550 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.501446    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89-tls-certs\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:01:57.547698 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.547670    2580 generic.go:356] "Generic (PLEG): container finished" podID="cdd50c6e-4658-4ca7-b6e8-1b024dce1f89" containerID="f5e6e943d76db54f0a263db6b7ef369e84097eb006ecb5fd9c363e1f1ff737b1" exitCode=0
Jul 29 22:01:57.547788 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.547714    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"
Jul 29 22:01:57.547788 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.547747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4" event={"ID":"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89","Type":"ContainerDied","Data":"f5e6e943d76db54f0a263db6b7ef369e84097eb006ecb5fd9c363e1f1ff737b1"}
Jul 29 22:01:57.547877 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.547787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4" event={"ID":"cdd50c6e-4658-4ca7-b6e8-1b024dce1f89","Type":"ContainerDied","Data":"e3930838d8865e21bdf8f27d2ddd07b482f6a85a3302bdb32158914a535412d9"}
Jul 29 22:01:57.547877 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.547805    2580 scope.go:117] "RemoveContainer" containerID="f5e6e943d76db54f0a263db6b7ef369e84097eb006ecb5fd9c363e1f1ff737b1"
Jul 29 22:01:57.556231 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.556215    2580 scope.go:117] "RemoveContainer" containerID="f5e6e943d76db54f0a263db6b7ef369e84097eb006ecb5fd9c363e1f1ff737b1"
Jul 29 22:01:57.556481 ip-10-0-138-99 kubenswrapper[2580]: E0729 22:01:57.556464    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5e6e943d76db54f0a263db6b7ef369e84097eb006ecb5fd9c363e1f1ff737b1\": container with ID starting with f5e6e943d76db54f0a263db6b7ef369e84097eb006ecb5fd9c363e1f1ff737b1 not found: ID does not exist" containerID="f5e6e943d76db54f0a263db6b7ef369e84097eb006ecb5fd9c363e1f1ff737b1"
Jul 29 22:01:57.556522 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.556489    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e6e943d76db54f0a263db6b7ef369e84097eb006ecb5fd9c363e1f1ff737b1"} err="failed to get container status \"f5e6e943d76db54f0a263db6b7ef369e84097eb006ecb5fd9c363e1f1ff737b1\": rpc error: code = NotFound desc = could not find container \"f5e6e943d76db54f0a263db6b7ef369e84097eb006ecb5fd9c363e1f1ff737b1\": container with ID starting with f5e6e943d76db54f0a263db6b7ef369e84097eb006ecb5fd9c363e1f1ff737b1 not found: ID does not exist"
Jul 29 22:01:57.572102 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.570758    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"]
Jul 29 22:01:57.577372 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:57.577351    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-a0dedf/ipp-route-a0dedf-kserve-666b44dd8c-hppd4"]
Jul 29 22:01:58.714918 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:01:58.714873    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdd50c6e-4658-4ca7-b6e8-1b024dce1f89" path="/var/lib/kubelet/pods/cdd50c6e-4658-4ca7-b6e8-1b024dce1f89/volumes"
Jul 29 22:02:34.036573 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.036542    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8"]
Jul 29 22:02:34.036987 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.036914    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="920a1d5a-b13c-48c9-819a-040a450aae3b" containerName="maas-api"
Jul 29 22:02:34.036987 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.036925    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="920a1d5a-b13c-48c9-819a-040a450aae3b" containerName="maas-api"
Jul 29 22:02:34.036987 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.036945    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd50c6e-4658-4ca7-b6e8-1b024dce1f89" containerName="main"
Jul 29 22:02:34.036987 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.036950    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd50c6e-4658-4ca7-b6e8-1b024dce1f89" containerName="main"
Jul 29 22:02:34.036987 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.036958    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b30364cd-d45e-4917-a926-c5d55df0dbd9" containerName="revoke-keys"
Jul 29 22:02:34.036987 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.036963    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b30364cd-d45e-4917-a926-c5d55df0dbd9" containerName="revoke-keys"
Jul 29 22:02:34.037222 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.037013    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="920a1d5a-b13c-48c9-819a-040a450aae3b" containerName="maas-api"
Jul 29 22:02:34.037222 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.037025    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b30364cd-d45e-4917-a926-c5d55df0dbd9" containerName="revoke-keys"
Jul 29 22:02:34.037222 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.037033    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd50c6e-4658-4ca7-b6e8-1b024dce1f89" containerName="main"
Jul 29 22:02:34.040059 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.040043    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8"
Jul 29 22:02:34.042604 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.042582    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9wjvq\""
Jul 29 22:02:34.047217 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.046925    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8"]
Jul 29 22:02:34.169051 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.169023    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb509e09-384c-451b-9121-2d2a57cefdb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8\" (UID: \"bb509e09-384c-451b-9121-2d2a57cefdb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8"
Jul 29 22:02:34.269643 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.269618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb509e09-384c-451b-9121-2d2a57cefdb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8\" (UID: \"bb509e09-384c-451b-9121-2d2a57cefdb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8"
Jul 29 22:02:34.270180 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.270164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb509e09-384c-451b-9121-2d2a57cefdb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8\" (UID: \"bb509e09-384c-451b-9121-2d2a57cefdb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8"
Jul 29 22:02:34.350872 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.350794    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8"
Jul 29 22:02:34.499895 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.499872    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8"]
Jul 29 22:02:34.502488 ip-10-0-138-99 kubenswrapper[2580]: W0729 22:02:34.502454    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb509e09_384c_451b_9121_2d2a57cefdb1.slice/crio-edea73c1af6526851008f9a9f55326cff9d7d0bb1654ace22192450fa30b974a WatchSource:0}: Error finding container edea73c1af6526851008f9a9f55326cff9d7d0bb1654ace22192450fa30b974a: Status 404 returned error can't find the container with id edea73c1af6526851008f9a9f55326cff9d7d0bb1654ace22192450fa30b974a
Jul 29 22:02:34.742508 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:34.742480    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8" event={"ID":"bb509e09-384c-451b-9121-2d2a57cefdb1","Type":"ContainerStarted","Data":"edea73c1af6526851008f9a9f55326cff9d7d0bb1654ace22192450fa30b974a"}
Jul 29 22:02:35.749756 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:35.749719    2580 generic.go:356] "Generic (PLEG): container finished" podID="bb509e09-384c-451b-9121-2d2a57cefdb1" containerID="cff46b4f9b2479fb7bee8035a4a0f68c9f1a527b21571cafc2853ac8e513cce1" exitCode=0
Jul 29 22:02:35.750191 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:35.749794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8" event={"ID":"bb509e09-384c-451b-9121-2d2a57cefdb1","Type":"ContainerDied","Data":"cff46b4f9b2479fb7bee8035a4a0f68c9f1a527b21571cafc2853ac8e513cce1"}
Jul 29 22:02:36.884763 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:36.884739    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8"
Jul 29 22:02:36.990447 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:36.990418    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb509e09-384c-451b-9121-2d2a57cefdb1-maas-api-service-ca\") pod \"bb509e09-384c-451b-9121-2d2a57cefdb1\" (UID: \"bb509e09-384c-451b-9121-2d2a57cefdb1\") "
Jul 29 22:02:36.990793 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:36.990766    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb509e09-384c-451b-9121-2d2a57cefdb1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bb509e09-384c-451b-9121-2d2a57cefdb1" (UID: "bb509e09-384c-451b-9121-2d2a57cefdb1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:02:37.091721 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:37.091658    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb509e09-384c-451b-9121-2d2a57cefdb1-maas-api-service-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:02:37.761268 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:37.761232    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8"
Jul 29 22:02:37.761443 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:37.761269    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8" event={"ID":"bb509e09-384c-451b-9121-2d2a57cefdb1","Type":"ContainerDied","Data":"edea73c1af6526851008f9a9f55326cff9d7d0bb1654ace22192450fa30b974a"}
Jul 29 22:02:37.761443 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:37.761334    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="edea73c1af6526851008f9a9f55326cff9d7d0bb1654ace22192450fa30b974a"
Jul 29 22:02:44.511921 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.511881    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"]
Jul 29 22:02:44.512496 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.512128    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf" podUID="0020d846-4a8a-487b-9fe6-d03832e098b3" containerName="maas-api" containerID="cri-o://a29d9c91b9dec90e89e440628910ea7fc0b4bedc302c9bcd43540955763f7aa7" gracePeriod=30
Jul 29 22:02:44.707709 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.707678    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8"]
Jul 29 22:02:44.714918 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.714893    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-50d50c-dzbd8"]
Jul 29 22:02:44.755940 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.755915    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 22:02:44.796171 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.796100    2580 generic.go:356] "Generic (PLEG): container finished" podID="0020d846-4a8a-487b-9fe6-d03832e098b3" containerID="a29d9c91b9dec90e89e440628910ea7fc0b4bedc302c9bcd43540955763f7aa7" exitCode=0
Jul 29 22:02:44.796171 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.796133    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"
Jul 29 22:02:44.796356 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.796184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf" event={"ID":"0020d846-4a8a-487b-9fe6-d03832e098b3","Type":"ContainerDied","Data":"a29d9c91b9dec90e89e440628910ea7fc0b4bedc302c9bcd43540955763f7aa7"}
Jul 29 22:02:44.796356 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.796248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf" event={"ID":"0020d846-4a8a-487b-9fe6-d03832e098b3","Type":"ContainerDied","Data":"0859aeb1059813256d9bdd54f4aa47bb50b2e47cf105089968d9aff23003bef1"}
Jul 29 22:02:44.796356 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.796278    2580 scope.go:117] "RemoveContainer" containerID="a29d9c91b9dec90e89e440628910ea7fc0b4bedc302c9bcd43540955763f7aa7"
Jul 29 22:02:44.804342 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.804326    2580 scope.go:117] "RemoveContainer" containerID="a29d9c91b9dec90e89e440628910ea7fc0b4bedc302c9bcd43540955763f7aa7"
Jul 29 22:02:44.804595 ip-10-0-138-99 kubenswrapper[2580]: E0729 22:02:44.804579    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a29d9c91b9dec90e89e440628910ea7fc0b4bedc302c9bcd43540955763f7aa7\": container with ID starting with a29d9c91b9dec90e89e440628910ea7fc0b4bedc302c9bcd43540955763f7aa7 not found: ID does not exist" containerID="a29d9c91b9dec90e89e440628910ea7fc0b4bedc302c9bcd43540955763f7aa7"
Jul 29 22:02:44.804645 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.804602    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a29d9c91b9dec90e89e440628910ea7fc0b4bedc302c9bcd43540955763f7aa7"} err="failed to get container status \"a29d9c91b9dec90e89e440628910ea7fc0b4bedc302c9bcd43540955763f7aa7\": rpc error: code = NotFound desc = could not find container \"a29d9c91b9dec90e89e440628910ea7fc0b4bedc302c9bcd43540955763f7aa7\": container with ID starting with a29d9c91b9dec90e89e440628910ea7fc0b4bedc302c9bcd43540955763f7aa7 not found: ID does not exist"
Jul 29 22:02:44.845414 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.845380    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0020d846-4a8a-487b-9fe6-d03832e098b3-openshift-service-ca\") pod \"0020d846-4a8a-487b-9fe6-d03832e098b3\" (UID: \"0020d846-4a8a-487b-9fe6-d03832e098b3\") "
Jul 29 22:02:44.845502 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.845483    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pqsr\" (UniqueName: \"kubernetes.io/projected/0020d846-4a8a-487b-9fe6-d03832e098b3-kube-api-access-7pqsr\") pod \"0020d846-4a8a-487b-9fe6-d03832e098b3\" (UID: \"0020d846-4a8a-487b-9fe6-d03832e098b3\") "
Jul 29 22:02:44.845593 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.845569    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0020d846-4a8a-487b-9fe6-d03832e098b3-maas-api-tls\") pod \"0020d846-4a8a-487b-9fe6-d03832e098b3\" (UID: \"0020d846-4a8a-487b-9fe6-d03832e098b3\") "
Jul 29 22:02:44.845769 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.845743    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0020d846-4a8a-487b-9fe6-d03832e098b3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0020d846-4a8a-487b-9fe6-d03832e098b3" (UID: "0020d846-4a8a-487b-9fe6-d03832e098b3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:02:44.845864 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.845821    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0020d846-4a8a-487b-9fe6-d03832e098b3-openshift-service-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:02:44.847426 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.847402    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0020d846-4a8a-487b-9fe6-d03832e098b3-kube-api-access-7pqsr" (OuterVolumeSpecName: "kube-api-access-7pqsr") pod "0020d846-4a8a-487b-9fe6-d03832e098b3" (UID: "0020d846-4a8a-487b-9fe6-d03832e098b3"). InnerVolumeSpecName "kube-api-access-7pqsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:02:44.847509 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.847474    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0020d846-4a8a-487b-9fe6-d03832e098b3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0020d846-4a8a-487b-9fe6-d03832e098b3" (UID: "0020d846-4a8a-487b-9fe6-d03832e098b3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:02:44.946326 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.946304    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0020d846-4a8a-487b-9fe6-d03832e098b3-maas-api-tls\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:02:44.946326 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:44.946324    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pqsr\" (UniqueName: \"kubernetes.io/projected/0020d846-4a8a-487b-9fe6-d03832e098b3-kube-api-access-7pqsr\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:02:45.117652 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:45.117628    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"]
Jul 29 22:02:45.124385 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:45.124363    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-50d50c-67db9dc748-p8xmf"]
Jul 29 22:02:46.716982 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:46.716943    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0020d846-4a8a-487b-9fe6-d03832e098b3" path="/var/lib/kubelet/pods/0020d846-4a8a-487b-9fe6-d03832e098b3/volumes"
Jul 29 22:02:46.717386 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:46.717268    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb509e09-384c-451b-9121-2d2a57cefdb1" path="/var/lib/kubelet/pods/bb509e09-384c-451b-9121-2d2a57cefdb1/volumes"
Jul 29 22:02:51.535935 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.535899    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf"]
Jul 29 22:02:51.536334 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.536231    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0020d846-4a8a-487b-9fe6-d03832e098b3" containerName="maas-api"
Jul 29 22:02:51.536334 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.536242    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0020d846-4a8a-487b-9fe6-d03832e098b3" containerName="maas-api"
Jul 29 22:02:51.536334 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.536256    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb509e09-384c-451b-9121-2d2a57cefdb1" containerName="revoke-keys"
Jul 29 22:02:51.536334 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.536261    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb509e09-384c-451b-9121-2d2a57cefdb1" containerName="revoke-keys"
Jul 29 22:02:51.536334 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.536318    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0020d846-4a8a-487b-9fe6-d03832e098b3" containerName="maas-api"
Jul 29 22:02:51.536334 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.536328    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb509e09-384c-451b-9121-2d2a57cefdb1" containerName="revoke-keys"
Jul 29 22:02:51.539338 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.539317    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf"
Jul 29 22:02:51.542215 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.542190    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 22:02:51.542322 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.542309    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9wjvq\""
Jul 29 22:02:51.546489 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.546461    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf"]
Jul 29 22:02:51.594598 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.594573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e60a09f2-2510-4ec1-b4ea-ce38c328c2f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf\" (UID: \"e60a09f2-2510-4ec1-b4ea-ce38c328c2f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf"
Jul 29 22:02:51.695045 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.695015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e60a09f2-2510-4ec1-b4ea-ce38c328c2f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf\" (UID: \"e60a09f2-2510-4ec1-b4ea-ce38c328c2f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf"
Jul 29 22:02:51.695565 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.695540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e60a09f2-2510-4ec1-b4ea-ce38c328c2f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf\" (UID: \"e60a09f2-2510-4ec1-b4ea-ce38c328c2f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf"
Jul 29 22:02:51.850384 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.850326    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf"
Jul 29 22:02:51.974346 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:51.974322    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf"]
Jul 29 22:02:51.976632 ip-10-0-138-99 kubenswrapper[2580]: W0729 22:02:51.976605    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode60a09f2_2510_4ec1_b4ea_ce38c328c2f4.slice/crio-3d5b5f6b6e3e0e0133fdb0e015f13b2720ea3213ba2e26b33d273dba13972374 WatchSource:0}: Error finding container 3d5b5f6b6e3e0e0133fdb0e015f13b2720ea3213ba2e26b33d273dba13972374: Status 404 returned error can't find the container with id 3d5b5f6b6e3e0e0133fdb0e015f13b2720ea3213ba2e26b33d273dba13972374
Jul 29 22:02:52.842582 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:52.842502    2580 generic.go:356] "Generic (PLEG): container finished" podID="e60a09f2-2510-4ec1-b4ea-ce38c328c2f4" containerID="196f2d1447d787132f1880c7852c8a4333d2a255d7654e747b37ebee1203f1e5" exitCode=0
Jul 29 22:02:52.842948 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:52.842603    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf" event={"ID":"e60a09f2-2510-4ec1-b4ea-ce38c328c2f4","Type":"ContainerDied","Data":"196f2d1447d787132f1880c7852c8a4333d2a255d7654e747b37ebee1203f1e5"}
Jul 29 22:02:52.842948 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:52.842651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf" event={"ID":"e60a09f2-2510-4ec1-b4ea-ce38c328c2f4","Type":"ContainerStarted","Data":"3d5b5f6b6e3e0e0133fdb0e015f13b2720ea3213ba2e26b33d273dba13972374"}
Jul 29 22:02:53.975722 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:53.975701    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf"
Jul 29 22:02:54.013286 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:54.013262    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e60a09f2-2510-4ec1-b4ea-ce38c328c2f4-maas-api-service-ca\") pod \"e60a09f2-2510-4ec1-b4ea-ce38c328c2f4\" (UID: \"e60a09f2-2510-4ec1-b4ea-ce38c328c2f4\") "
Jul 29 22:02:54.013574 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:54.013554    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e60a09f2-2510-4ec1-b4ea-ce38c328c2f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e60a09f2-2510-4ec1-b4ea-ce38c328c2f4" (UID: "e60a09f2-2510-4ec1-b4ea-ce38c328c2f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:02:54.114393 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:54.114336    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e60a09f2-2510-4ec1-b4ea-ce38c328c2f4-maas-api-service-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:02:54.853733 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:54.853696    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf"
Jul 29 22:02:54.853733 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:54.853709    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf" event={"ID":"e60a09f2-2510-4ec1-b4ea-ce38c328c2f4","Type":"ContainerDied","Data":"3d5b5f6b6e3e0e0133fdb0e015f13b2720ea3213ba2e26b33d273dba13972374"}
Jul 29 22:02:54.853940 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:02:54.853757    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d5b5f6b6e3e0e0133fdb0e015f13b2720ea3213ba2e26b33d273dba13972374"
Jul 29 22:03:02.232797 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:02.232762    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf"]
Jul 29 22:03:02.235740 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:02.235715    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fe32bc-k7lvf"]
Jul 29 22:03:02.719747 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:02.719715    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e60a09f2-2510-4ec1-b4ea-ce38c328c2f4" path="/var/lib/kubelet/pods/e60a09f2-2510-4ec1-b4ea-ce38c328c2f4/volumes"
Jul 29 22:03:09.300583 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.300504    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7"]
Jul 29 22:03:09.300958 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.300890    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e60a09f2-2510-4ec1-b4ea-ce38c328c2f4" containerName="revoke-keys"
Jul 29 22:03:09.300958 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.300903    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e60a09f2-2510-4ec1-b4ea-ce38c328c2f4" containerName="revoke-keys"
Jul 29 22:03:09.300958 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.300958    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e60a09f2-2510-4ec1-b4ea-ce38c328c2f4" containerName="revoke-keys"
Jul 29 22:03:09.305138 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.305121    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7"
Jul 29 22:03:09.308183 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.308162    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 22:03:09.308298 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.308209    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9wjvq\""
Jul 29 22:03:09.311743 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.311719    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7"]
Jul 29 22:03:09.429682 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.429653    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6717b712-c935-4720-9374-bab24795dd6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7\" (UID: \"6717b712-c935-4720-9374-bab24795dd6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7"
Jul 29 22:03:09.530810 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.530779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6717b712-c935-4720-9374-bab24795dd6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7\" (UID: \"6717b712-c935-4720-9374-bab24795dd6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7"
Jul 29 22:03:09.531373 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.531348    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6717b712-c935-4720-9374-bab24795dd6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7\" (UID: \"6717b712-c935-4720-9374-bab24795dd6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7"
Jul 29 22:03:09.617392 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.617332    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7"
Jul 29 22:03:09.741004 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.740978    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7"]
Jul 29 22:03:09.743171 ip-10-0-138-99 kubenswrapper[2580]: W0729 22:03:09.743145    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6717b712_c935_4720_9374_bab24795dd6f.slice/crio-86ffffbbfae29d536cb2052f2569bdbb4f0a21ba49b2b098dc3ba01e3c328778 WatchSource:0}: Error finding container 86ffffbbfae29d536cb2052f2569bdbb4f0a21ba49b2b098dc3ba01e3c328778: Status 404 returned error can't find the container with id 86ffffbbfae29d536cb2052f2569bdbb4f0a21ba49b2b098dc3ba01e3c328778
Jul 29 22:03:09.939452 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:09.939420    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7" event={"ID":"6717b712-c935-4720-9374-bab24795dd6f","Type":"ContainerStarted","Data":"86ffffbbfae29d536cb2052f2569bdbb4f0a21ba49b2b098dc3ba01e3c328778"}
Jul 29 22:03:10.946305 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:10.946271    2580 generic.go:356] "Generic (PLEG): container finished" podID="6717b712-c935-4720-9374-bab24795dd6f" containerID="642ab53469fc7bd7e4779108cd9bac8e090d15f67a8b8f7c931b79d74bd63064" exitCode=0
Jul 29 22:03:10.946705 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:10.946323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7" event={"ID":"6717b712-c935-4720-9374-bab24795dd6f","Type":"ContainerDied","Data":"642ab53469fc7bd7e4779108cd9bac8e090d15f67a8b8f7c931b79d74bd63064"}
Jul 29 22:03:12.080434 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:12.080412    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7"
Jul 29 22:03:12.148050 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:12.148027    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6717b712-c935-4720-9374-bab24795dd6f-maas-api-service-ca\") pod \"6717b712-c935-4720-9374-bab24795dd6f\" (UID: \"6717b712-c935-4720-9374-bab24795dd6f\") "
Jul 29 22:03:12.148339 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:12.148317    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6717b712-c935-4720-9374-bab24795dd6f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6717b712-c935-4720-9374-bab24795dd6f" (UID: "6717b712-c935-4720-9374-bab24795dd6f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:03:12.248872 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:12.248803    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6717b712-c935-4720-9374-bab24795dd6f-maas-api-service-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\""
Jul 29 22:03:12.957333 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:12.957294    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7" event={"ID":"6717b712-c935-4720-9374-bab24795dd6f","Type":"ContainerDied","Data":"86ffffbbfae29d536cb2052f2569bdbb4f0a21ba49b2b098dc3ba01e3c328778"}
Jul 29 22:03:12.957481 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:12.957346    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86ffffbbfae29d536cb2052f2569bdbb4f0a21ba49b2b098dc3ba01e3c328778"
Jul 29 22:03:12.957481 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:12.957302    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7"
Jul 29 22:03:19.782913 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:19.782871    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7"]
Jul 29 22:03:19.785448 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:19.785414    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-670ddc-29756040-4xgk7"]
Jul 29 22:03:19.980422 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:19.980388    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7"]
Jul 29 22:03:19.982740 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:19.982717    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-670ddc-7d2p7"]
Jul 29 22:03:20.715737 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:20.715693    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6717b712-c935-4720-9374-bab24795dd6f" path="/var/lib/kubelet/pods/6717b712-c935-4720-9374-bab24795dd6f/volumes"
Jul 29 22:03:20.716097 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:20.716073    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed03e702-032f-411c-87b3-4c336bf97424" path="/var/lib/kubelet/pods/ed03e702-032f-411c-87b3-4c336bf97424/volumes"
Jul 29 22:03:34.081032 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:34.081003    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8446ffcd9c-zc97t_5eefeb19-409d-4122-a96a-11db885cbb13/copy-manifests/0.log"
Jul 29 22:03:34.087875 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:34.087849    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8446ffcd9c-zc97t_5eefeb19-409d-4122-a96a-11db885cbb13/manager/0.log"
Jul 29 22:03:37.740758 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:37.740723    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hgj5f_2598bc39-4fdd-4108-a10e-3b79e10fc161/discovery/0.log"
Jul 29 22:03:37.864518 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:37.864491    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75d94d9cdd-2dbzv_9a389da2-ad1f-442a-aa1f-5727e3baa007/kube-auth-proxy/0.log"
Jul 29 22:03:38.502800 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:38.502775    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5dd67cb7f-4wd8g_f314863c-51a2-4c85-aa0b-96f18234f702/router/0.log"
Jul 29 22:03:48.569947 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:48.569909    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-phqpp_f3c9b5cc-c1df-42ab-81d3-ee6dc4b1b9a3/global-pull-secret-syncer/0.log"
Jul 29 22:03:48.747944 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:48.747903    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sxpkh_308b710d-1f57-4abb-9d06-273683523857/konnectivity-agent/0.log"
Jul 29 22:03:48.794583 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:48.794559    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-99.ec2.internal_a6d41c68b970bb763324c8f000cd0b6e/haproxy/0.log"
Jul 29 22:03:55.631463 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:55.631423    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-mqzbx_3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6/kube-state-metrics/0.log"
Jul 29 22:03:55.650213 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:55.650186    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-mqzbx_3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6/kube-rbac-proxy-main/0.log"
Jul 29 22:03:55.672585 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:55.672560    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-mqzbx_3dd3f78f-08fe-4f4d-a3ed-39d3eead0aa6/kube-rbac-proxy-self/0.log"
Jul 29 22:03:55.699955 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:55.699935    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6bcfbcc6cc-mb5zt_fe1085ac-7489-457c-aa7c-cffe927bf9b1/metrics-server/0.log"
Jul 29 22:03:55.883023 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:55.882954    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cwplq_00abefd7-4870-4e91-b171-2dfe9b4db905/node-exporter/0.log"
Jul 29 22:03:55.908945 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:55.908923    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cwplq_00abefd7-4870-4e91-b171-2dfe9b4db905/kube-rbac-proxy/0.log"
Jul 29 22:03:55.932197 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:55.932179    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cwplq_00abefd7-4870-4e91-b171-2dfe9b4db905/init-textfile/0.log"
Jul 29 22:03:56.372021 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:56.371995    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-rtxpt_1731f9aa-3532-4995-bf66-c5f92137190d/prometheus-operator/0.log"
Jul 29 22:03:56.393386 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:56.393366    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-rtxpt_1731f9aa-3532-4995-bf66-c5f92137190d/kube-rbac-proxy/0.log"
Jul 29 22:03:56.545652 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:56.545630    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74df9db688-f58fb_c9e6fcbb-ee46-47d2-a409-e4748c66ce90/thanos-query/0.log"
Jul 29 22:03:56.566085 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:56.566065    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74df9db688-f58fb_c9e6fcbb-ee46-47d2-a409-e4748c66ce90/kube-rbac-proxy-web/0.log"
Jul 29 22:03:56.592443 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:56.592422    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74df9db688-f58fb_c9e6fcbb-ee46-47d2-a409-e4748c66ce90/kube-rbac-proxy/0.log"
Jul 29 22:03:56.613194 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:56.613178    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74df9db688-f58fb_c9e6fcbb-ee46-47d2-a409-e4748c66ce90/prom-label-proxy/0.log"
Jul 29 22:03:56.633870 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:56.633831    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74df9db688-f58fb_c9e6fcbb-ee46-47d2-a409-e4748c66ce90/kube-rbac-proxy-rules/0.log"
Jul 29 22:03:56.661304 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:56.661285    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74df9db688-f58fb_c9e6fcbb-ee46-47d2-a409-e4748c66ce90/kube-rbac-proxy-metrics/0.log"
Jul 29 22:03:58.680312 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.680278    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 22:03:58.685320 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.685259    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/3.log"
Jul 29 22:03:58.800691 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.800660    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"]
Jul 29 22:03:58.801002 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.800988    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6717b712-c935-4720-9374-bab24795dd6f" containerName="revoke-keys"
Jul 29 22:03:58.801064 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.801003    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6717b712-c935-4720-9374-bab24795dd6f" containerName="revoke-keys"
Jul 29 22:03:58.801104 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.801064    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6717b712-c935-4720-9374-bab24795dd6f" containerName="revoke-keys"
Jul 29 22:03:58.804194 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.804174    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.807013 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.806985    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xq8zq\"/\"openshift-service-ca.crt\""
Jul 29 22:03:58.807129 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.807041    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xq8zq\"/\"kube-root-ca.crt\""
Jul 29 22:03:58.807192 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.807133    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xq8zq\"/\"default-dockercfg-9jqsr\""
Jul 29 22:03:58.814010 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.813983    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"]
Jul 29 22:03:58.893167 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.893143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50bc389b-684a-4b97-96d4-098dfe2e84d2-lib-modules\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.893270 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.893175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/50bc389b-684a-4b97-96d4-098dfe2e84d2-proc\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.893270 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.893192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lzs4\" (UniqueName: \"kubernetes.io/projected/50bc389b-684a-4b97-96d4-098dfe2e84d2-kube-api-access-4lzs4\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.893270 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.893212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50bc389b-684a-4b97-96d4-098dfe2e84d2-sys\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.893270 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.893232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/50bc389b-684a-4b97-96d4-098dfe2e84d2-podres\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.993772 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.993705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/50bc389b-684a-4b97-96d4-098dfe2e84d2-podres\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.993772 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.993768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50bc389b-684a-4b97-96d4-098dfe2e84d2-lib-modules\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.993969 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.993798    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/50bc389b-684a-4b97-96d4-098dfe2e84d2-proc\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.993969 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.993817    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lzs4\" (UniqueName: \"kubernetes.io/projected/50bc389b-684a-4b97-96d4-098dfe2e84d2-kube-api-access-4lzs4\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.993969 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.993865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50bc389b-684a-4b97-96d4-098dfe2e84d2-sys\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.993969 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.993924    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/50bc389b-684a-4b97-96d4-098dfe2e84d2-podres\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.993969 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.993940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50bc389b-684a-4b97-96d4-098dfe2e84d2-sys\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.993969 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.993929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/50bc389b-684a-4b97-96d4-098dfe2e84d2-proc\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:58.994186 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:58.993964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50bc389b-684a-4b97-96d4-098dfe2e84d2-lib-modules\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:59.003069 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:59.003050    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lzs4\" (UniqueName: \"kubernetes.io/projected/50bc389b-684a-4b97-96d4-098dfe2e84d2-kube-api-access-4lzs4\") pod \"perf-node-gather-daemonset-ld2qm\" (UID: \"50bc389b-684a-4b97-96d4-098dfe2e84d2\") " pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:59.115339 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:59.115314    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:03:59.248284 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:59.248263    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"]
Jul 29 22:03:59.250672 ip-10-0-138-99 kubenswrapper[2580]: W0729 22:03:59.250648    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod50bc389b_684a_4b97_96d4_098dfe2e84d2.slice/crio-1d51017b71e5881ea66216881692b1974bb0ada95cb439c36266b25f718b1f03 WatchSource:0}: Error finding container 1d51017b71e5881ea66216881692b1974bb0ada95cb439c36266b25f718b1f03: Status 404 returned error can't find the container with id 1d51017b71e5881ea66216881692b1974bb0ada95cb439c36266b25f718b1f03
Jul 29 22:03:59.300502 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:03:59.300482    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-jrpkc_3433dc8c-6a78-4170-8c53-48d19c2ff6d3/download-server/0.log"
Jul 29 22:04:00.196077 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:00.195995    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm" event={"ID":"50bc389b-684a-4b97-96d4-098dfe2e84d2","Type":"ContainerStarted","Data":"a27b54ced5f989ac195712bf4e381949bf291e7645ccb57bcc9f79d79b3fa9e8"}
Jul 29 22:04:00.196450 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:00.196126    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:04:00.196450 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:00.196184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm" event={"ID":"50bc389b-684a-4b97-96d4-098dfe2e84d2","Type":"ContainerStarted","Data":"1d51017b71e5881ea66216881692b1974bb0ada95cb439c36266b25f718b1f03"}
Jul 29 22:04:00.213497 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:00.213446    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm" podStartSLOduration=2.213422964 podStartE2EDuration="2.213422964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:03:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:04:00.211479681 +0000 UTC m=+3592.092753818" watchObservedRunningTime="2026-07-29 22:04:00.213422964 +0000 UTC m=+3592.094697101"
Jul 29 22:04:00.988919 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:00.988890    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7hv9z_17ed28ff-e083-4d5e-819c-142ff66b85dd/dns/0.log"
Jul 29 22:04:01.010221 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:01.010196    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7hv9z_17ed28ff-e083-4d5e-819c-142ff66b85dd/kube-rbac-proxy/0.log"
Jul 29 22:04:01.180021 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:01.179992    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f58qv_97bd114e-b39c-4454-be0a-a964b2db56f8/dns-node-resolver/0.log"
Jul 29 22:04:01.739304 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:01.739281    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-25rxb_d44ed47a-d0fe-4536-a552-4ee051c80914/node-ca/0.log"
Jul 29 22:04:02.869267 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:02.869244    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hgj5f_2598bc39-4fdd-4108-a10e-3b79e10fc161/discovery/0.log"
Jul 29 22:04:02.889910 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:02.889886    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75d94d9cdd-2dbzv_9a389da2-ad1f-442a-aa1f-5727e3baa007/kube-auth-proxy/0.log"
Jul 29 22:04:03.072123 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:03.072096    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5dd67cb7f-4wd8g_f314863c-51a2-4c85-aa0b-96f18234f702/router/0.log"
Jul 29 22:04:03.993052 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:03.993028    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lwwqd_f60fcc10-a90b-4f77-bd05-f23f5e94fa69/serve-healthcheck-canary/0.log"
Jul 29 22:04:04.613288 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:04.613260    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j2scg_88e24522-aa6c-43bb-8805-b8e4d682e5a2/kube-rbac-proxy/0.log"
Jul 29 22:04:04.635511 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:04.635482    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j2scg_88e24522-aa6c-43bb-8805-b8e4d682e5a2/exporter/0.log"
Jul 29 22:04:04.659275 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:04.659250    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j2scg_88e24522-aa6c-43bb-8805-b8e4d682e5a2/extractor/0.log"
Jul 29 22:04:06.210140 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:06.210118    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xq8zq/perf-node-gather-daemonset-ld2qm"
Jul 29 22:04:07.521653 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:07.521628    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8446ffcd9c-zc97t_5eefeb19-409d-4122-a96a-11db885cbb13/manager/0.log"
Jul 29 22:04:07.543717 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:07.543697    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8446ffcd9c-zc97t_5eefeb19-409d-4122-a96a-11db885cbb13/copy-manifests/0.log"
Jul 29 22:04:08.826980 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:08.826901    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 22:04:08.836413 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:08.831141    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 22:04:08.839930 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:08.839912    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ssvr7_42ac4b00-61be-4468-8b6d-e85603e33b67/console-operator/2.log"
Jul 29 22:04:08.843397 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:08.843380    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4bvn_ea9e9d47-2e06-42bb-a685-e0e07f9d121c/ovn-acl-logging/0.log"
Jul 29 22:04:08.916680 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:08.916658    2580 scope.go:117] "RemoveContainer" containerID="16175bd1e6f0bef47e76c721b661e7e7088094142d2cb1fa14f5257a2e0daeb6"
Jul 29 22:04:08.971291 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:08.971275    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d97fdbb4b-glrwk_8ee44a3c-344b-48b8-b9de-a59be3c58684/manager/0.log"
Jul 29 22:04:09.628749 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:09.628718    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hgj5f_2598bc39-4fdd-4108-a10e-3b79e10fc161/discovery/0.log"
Jul 29 22:04:09.640052 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:09.640025    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75d94d9cdd-2dbzv_9a389da2-ad1f-442a-aa1f-5727e3baa007/kube-auth-proxy/0.log"
Jul 29 22:04:09.782740 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:09.782719    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5dd67cb7f-4wd8g_f314863c-51a2-4c85-aa0b-96f18234f702/router/0.log"
Jul 29 22:04:11.290102 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:11.290073    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hgj5f_2598bc39-4fdd-4108-a10e-3b79e10fc161/discovery/0.log"
Jul 29 22:04:11.304380 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:11.304343    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75d94d9cdd-2dbzv_9a389da2-ad1f-442a-aa1f-5727e3baa007/kube-auth-proxy/0.log"
Jul 29 22:04:11.435169 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:11.435146    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5dd67cb7f-4wd8g_f314863c-51a2-4c85-aa0b-96f18234f702/router/0.log"
Jul 29 22:04:14.353974 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:14.353943    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg_d135c4fb-0ebd-485a-8dbb-39369e431db7/kube-storage-version-migrator-operator/1.log"
Jul 29 22:04:14.354716 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:14.354693    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-g8fmg_d135c4fb-0ebd-485a-8dbb-39369e431db7/kube-storage-version-migrator-operator/0.log"
Jul 29 22:04:15.292965 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:15.292934    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jk6th_7a5822af-8e5a-4b6b-9797-38c1b32049b8/kube-multus-additional-cni-plugins/0.log"
Jul 29 22:04:15.314468 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:15.314446    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jk6th_7a5822af-8e5a-4b6b-9797-38c1b32049b8/egress-router-binary-copy/0.log"
Jul 29 22:04:15.335132 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:15.335112    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jk6th_7a5822af-8e5a-4b6b-9797-38c1b32049b8/cni-plugins/0.log"
Jul 29 22:04:15.365246 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:15.365227    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jk6th_7a5822af-8e5a-4b6b-9797-38c1b32049b8/bond-cni-plugin/0.log"
Jul 29 22:04:15.387454 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:15.387431    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jk6th_7a5822af-8e5a-4b6b-9797-38c1b32049b8/routeoverride-cni/0.log"
Jul 29 22:04:15.409770 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:15.409751    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jk6th_7a5822af-8e5a-4b6b-9797-38c1b32049b8/whereabouts-cni-bincopy/0.log"
Jul 29 22:04:15.431721 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:15.431705    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jk6th_7a5822af-8e5a-4b6b-9797-38c1b32049b8/whereabouts-cni/0.log"
Jul 29 22:04:15.801747 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:15.801719    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cxq9x_b3f9485e-51c3-4074-b7ed-aaf6aa491811/kube-multus/0.log"
Jul 29 22:04:15.990111 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:15.990082    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6m4sz_ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b/network-metrics-daemon/0.log"
Jul 29 22:04:16.009931 ip-10-0-138-99 kubenswrapper[2580]: I0729 22:04:16.009911    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6m4sz_ce5211e6-dfbb-42bb-a7ee-a63bb3087a1b/kube-rbac-proxy/0.log"