Jul 21 12:49:08.569159 ip-10-0-128-74 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 21 12:49:08.569171 ip-10-0-128-74 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 21 12:49:08.569180 ip-10-0-128-74 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 21 12:49:08.569538 ip-10-0-128-74 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 21 12:49:18.598306 ip-10-0-128-74 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 21 12:49:18.598327 ip-10-0-128-74 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 9a79e254de2a41b593c2922a5149cc7d --
Jul 21 12:51:05.970268 ip-10-0-128-74 systemd[1]: Starting Kubernetes Kubelet...
Jul 21 12:51:06.416865 ip-10-0-128-74 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 12:51:06.416865 ip-10-0-128-74 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 21 12:51:06.416865 ip-10-0-128-74 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 12:51:06.416865 ip-10-0-128-74 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 21 12:51:06.416865 ip-10-0-128-74 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 12:51:06.419866 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.419779    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 21 12:51:06.424506 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424481    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:51:06.424506 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424504    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:51:06.424506 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424510    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424515    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424520    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424524    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424528    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424532    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424536    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424539    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424543    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424547    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424550    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424554    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424558    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424562    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424575    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424580    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424583    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424589    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424594    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:51:06.424733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424599    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424603    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424608    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424612    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424616    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424620    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424623    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424628    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424632    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424636    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424642    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424646    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424650    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424654    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424659    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424664    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424668    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424672    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424677    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424681    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:51:06.425504 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424687    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424691    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424696    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424700    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424704    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424708    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424712    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424716    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424721    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424725    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424729    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424732    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424737    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424743    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424747    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424751    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424755    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424760    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424764    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424771    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:51:06.426370 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424776    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424781    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424785    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424789    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424793    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424798    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424803    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424813    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424817    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424821    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424826    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424830    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424834    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424839    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424843    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424847    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424852    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424856    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424860    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:51:06.427187 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424864    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424868    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424872    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424877    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424881    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424886    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424890    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424894    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424898    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424902    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424906    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424911    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424915    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424919    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424923    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424927    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424931    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424935    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.424939    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:51:06.427946 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426184    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426195    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426199    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426204    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426208    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426213    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426223    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426228    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426233    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426238    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426242    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426246    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426250    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426254    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426261    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426267    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426272    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426277    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426282    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426287    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:51:06.428751 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426291    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426296    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426301    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426306    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426310    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426317    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426323    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426328    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426331    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426336    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426341    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426345    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426350    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426354    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426359    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426364    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426368    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426372    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426376    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426380    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:51:06.429550 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426384    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426389    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426393    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426397    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426403    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426409    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426413    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426417    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426421    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426426    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426431    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426437    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426441    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426445    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426451    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426456    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426461    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426465    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426469    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:51:06.430074 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426473    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426477    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426481    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426486    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426490    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426494    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426498    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426503    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426507    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426511    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426515    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426519    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426524    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426528    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426532    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426537    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426542    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426546    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426550    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426554    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:51:06.430537 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426558    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426561    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426565    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426570    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426575    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426579    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426582    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426586    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426590    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426594    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426598    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426602    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426606    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426610    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426614    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426618    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426623    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426627    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426631    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.426635    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:51:06.431103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427294    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427304    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427309    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427313    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427318    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427322    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427327    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427331    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427336    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427341    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427345    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427349    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427354    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427359    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427363    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427367    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427372    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427376    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427380    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427385    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:51:06.431815 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427389    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427393    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427400    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427404    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427408    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427412    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427416    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427421    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427425    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427429    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427434    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427438    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427442    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427446    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427450    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427455    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427459    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427463    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427468    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427472    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:51:06.432308 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427477    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427481    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427484    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427488    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427492    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427496    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427500    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427505    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427509    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427513    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427517    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427521    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427525    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427529    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427535    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427539    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427543    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427547    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427551    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427555    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:51:06.432780 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427560    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427566    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427570    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427577    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427582    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427586    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427591    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427595    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427599    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427604    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427607    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427611    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427616    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427623    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427629    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427634    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427640    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427646    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:51:06.433361 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427651    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427658    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427665    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427670    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427674    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427678    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427682    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427686    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427692    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427697    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427701    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427705    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427710    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427714    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427718    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427721    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427726    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427730    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427735    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427740    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:51:06.433786 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.427744    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427843    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427866    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427877    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427884    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427891    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427896    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427903    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427910    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427916    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427921    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427926    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427931    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427938    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427942    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427947    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427951    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427973    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427978    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427985    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427992    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.427997    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428002    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428007    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 21 12:51:06.434299 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428014    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428019    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428024    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428029    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428034    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428039    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428044    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428049    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428054    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428060    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428065    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428069    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428074    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428079    2563 flags.go:64] FLAG: --enable-server="true"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428084    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428092    2563 flags.go:64] FLAG: --event-burst="100"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428096    2563 flags.go:64] FLAG: --event-qps="50"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428101    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428106    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428110    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428121    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428127    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428131    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428136    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428141    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 21 12:51:06.434846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428146    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428150    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428155    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428161    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428165    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428170    2563 flags.go:64] FLAG: --feature-gates=""
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428176    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428181    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428186    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428191    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428195    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428200    2563 flags.go:64] FLAG: --help="false"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428205    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428210    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428214    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428219    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428224    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428230    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428234    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428238    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428243    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428248    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428252    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428257    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 21 12:51:06.435586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428262    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428268    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428272    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428277    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428282    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428287    2563 flags.go:64] FLAG: --lock-file=""
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428291    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428296    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428301    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428309    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428321    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428328    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428332    2563 flags.go:64] FLAG: --logging-format="text"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428337    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428342    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428346    2563 flags.go:64] FLAG: --manifest-url=""
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428351    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428358    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428363    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428370    2563 flags.go:64] FLAG: --max-pods="110"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428374    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428379    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428383    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428388    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428392    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 21 12:51:06.436239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428397    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428401    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428413    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428418    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428423    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428428    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428433    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428442    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428447    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428453    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428457    2563 flags.go:64] FLAG: --port="10250"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428462    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428467    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ff14ed8495360009"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428472    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428476    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428481    2563 flags.go:64] FLAG: --register-node="true"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428486    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428491    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428496    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428502    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428507    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428513    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428518    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428523    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428528    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 21 12:51:06.436832 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428533    2563 flags.go:64] FLAG: --runonce="false"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428538    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428543    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428547    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428552    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428556    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428561    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428565    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428570    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428575    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428580    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428584    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428589    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428593    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428598    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428602    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428611    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428616    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428621    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428627    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428632    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428636    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428641    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428645    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428650    2563 flags.go:64] FLAG: --v="2"
Jul 21 12:51:06.437801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428657    2563 flags.go:64] FLAG: --version="false"
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428665    2563 flags.go:64] FLAG: --vmodule=""
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428671    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.428679    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428909    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428917    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428922    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428926    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428931    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428935    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428939    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428943    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428948    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428952    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428973    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428978    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428982    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428986    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428991    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428995    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:51:06.438510 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.428999    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429003    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429007    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429011    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429015    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429020    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429025    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429029    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429033    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429037    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429041    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429045    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429049    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429058    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429063    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429070    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429074    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429078    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429082    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429086    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:51:06.439079 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429090    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429094    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429098    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429102    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429107    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429111    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429114    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429119    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429123    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429127    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429131    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429135    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429139    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429143    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429147    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429151    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429155    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429159    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429164    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429168    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429173    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:51:06.439545 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429176    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429180    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429184    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429188    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429194    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429199    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429205    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429209    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429213    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429219    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429226    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429231    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429235    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429239    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429243    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429247    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429251    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429255    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429260    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:51:06.440103 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429264    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429268    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429271    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429275    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429279    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429283    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429288    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429292    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429296    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429300    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429304    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429309    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429313    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429317    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429321    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429326    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429329    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429336    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429340    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:51:06.440555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429348    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429354    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429360    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.429365    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.430565    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.439055    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.439072    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.439090    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.439100    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439156    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439161    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439165    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439167    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439171    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:51:06.441006 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439174    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439176    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439179    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439182    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439185    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439188    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439191    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439193    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439196    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439199    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439201    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439204    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439207    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439209    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439212    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439214    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439217    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439219    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439222    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439224    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:51:06.441357 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439227    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439230    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439232    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439235    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439239    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439241    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439244    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439247    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439250    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439252    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439255    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439258    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439261    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439263    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439266    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439269    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439271    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439274    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439276    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439279    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:51:06.441814 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439281    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439283    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439286    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439288    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439291    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439293    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439296    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439298    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439301    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439304    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439306    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439310    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439314    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439317    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439320    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439323    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439327    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439330    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439333    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439336    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:51:06.442296 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439339    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439342    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439345    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439347    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439350    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439352    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439356    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439359    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439363    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439367    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439370    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439372    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439375    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439377    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439380    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439383    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439386    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439388    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439391    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:51:06.442750 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439393    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439396    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439398    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439401    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439404    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439406    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439409    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439411    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439414    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439417    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439419    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439422    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439425    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439427    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439430    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:51:06.443248 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.439435    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439547    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439552    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439555    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439559    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439562    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439564    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439567    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439571    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439576    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439579    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439582    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439585    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439588    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439592    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439594    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439597    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439599    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439602    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439604    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:51:06.443598 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439607    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439609    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439612    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439614    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439617    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439619    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439622    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439625    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439628    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439630    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439633    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439636    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439638    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439641    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439644    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439648    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439651    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439654    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439656    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439659    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:51:06.444147 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439662    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439664    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439667    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439669    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439672    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439675    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439678    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439681    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439683    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439686    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439688    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439691    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439693    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439696    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439698    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439700    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439703    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439705    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439708    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439711    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:51:06.444616 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439713    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439716    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439718    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439720    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439723    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439726    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439728    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439731    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439733    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439735    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439738    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439740    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439743    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439745    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439749    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439752    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439755    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439758    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439761    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:51:06.445112 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439764    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439766    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439769    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439771    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439773    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439776    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439778    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439781    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439784    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439786    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439788    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439791    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439794    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439797    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439799    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439802    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439804    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439807    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439810    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439812    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:51:06.445554 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:06.439814    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:51:06.446024 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.439819    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 12:51:06.446024 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.439933    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 21 12:51:06.446024 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.442020    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 21 12:51:06.446024 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.443609    2563 server.go:1019] "Starting client certificate rotation"
Jul 21 12:51:06.446024 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.443710    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 12:51:06.446024 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.443742    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 12:51:06.446395 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.446380    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 12:51:06.447664 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.447647    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 12:51:06.462005 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.461982    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 21 12:51:06.464114 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.464101    2563 log.go:25] "Validated CRI v1 image API"
Jul 21 12:51:06.464980 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.464951    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 21 12:51:06.471829 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.471805    2563 fs.go:135] Filesystem UUIDs: map[4350bf85-cc2b-41f6-8437-b9583739fe3b:/dev/nvme0n1p4 711ac13a-c568-4459-9196-108ffaf1b4bd:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 21 12:51:06.471905 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.471826    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 21 12:51:06.475205 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.475188    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 12:51:06.477623 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.477497    2563 manager.go:216] Machine: {Timestamp:2026-07-21 12:51:06.476151106 +0000 UTC m=+0.381986327 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3195766 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ee545090dbcd32fe300ab74dc104b SystemUUID:ec2ee545-090d-bcd3-2fe3-00ab74dc104b BootID:9a79e254-de2a-41b5-93c2-922a5149cc7d Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5b:29:3d:c7:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5b:29:3d:c7:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:07:1f:d2:bf:68 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 21 12:51:06.477623 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.477618    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 21 12:51:06.477723 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.477701    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 21 12:51:06.482155 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.481983    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 21 12:51:06.482328 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.482157    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-74.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 21 12:51:06.483088 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.483076    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 21 12:51:06.483142 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.483093    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 21 12:51:06.483142 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.483103    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 12:51:06.483780 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.483767    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 12:51:06.485120 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.485103    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 21 12:51:06.488987 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.488975    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 12:51:06.489157 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.489148    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 21 12:51:06.489194 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.489178    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 21 12:51:06.489194 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.489187    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 21 12:51:06.489247 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.489204    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 21 12:51:06.489247 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.489214    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 21 12:51:06.489247 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.489223    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 21 12:51:06.489603 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.489592    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 12:51:06.489645 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.489610    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 12:51:06.492791 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.492770    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 21 12:51:06.494679 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.494663    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 21 12:51:06.494756 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.494700    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 21 12:51:06.495875 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.495862    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 21 12:51:06.495940 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.495883    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 21 12:51:06.495940 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.495892    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 21 12:51:06.495940 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.495901    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 21 12:51:06.495940 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.495914    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 21 12:51:06.495940 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.495922    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 21 12:51:06.495940 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.495934    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 21 12:51:06.495940 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.495942    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 21 12:51:06.496180 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.495952    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 21 12:51:06.496180 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.495979    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 21 12:51:06.496180 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.495993    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 21 12:51:06.496180 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.496002    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 21 12:51:06.497487 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.497473    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 21 12:51:06.497548 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.497490    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 21 12:51:06.499622 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.499585    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 21 12:51:06.499622 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.499607    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-74.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 21 12:51:06.499821 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.499795    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-74.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 21 12:51:06.500722 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.500706    2563 server.go:1268] "Started kubelet"
Jul 21 12:51:06.500872 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.500775    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 21 12:51:06.500949 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.500850    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 21 12:51:06.501078 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.500992    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 21 12:51:06.501546 ip-10-0-128-74 systemd[1]: Started Kubernetes Kubelet.
Jul 21 12:51:06.503617 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.502730    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 21 12:51:06.504073 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.504050    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 21 12:51:06.505342 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.505312    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bnsst"
Jul 21 12:51:06.507925 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.507902    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 21 12:51:06.510378 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.510355    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 21 12:51:06.511119 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.511102    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 21 12:51:06.511227 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.511208    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-74.ec2.internal\" not found"
Jul 21 12:51:06.511876 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.511856    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 21 12:51:06.511876 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.511856    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 21 12:51:06.512042 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.511887    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 21 12:51:06.512042 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.511939    2563 factory.go:55] Registering systemd factory
Jul 21 12:51:06.512042 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.511955    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 21 12:51:06.512042 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.512011    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 21 12:51:06.512042 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.512018    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 21 12:51:06.512239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.512226    2563 factory.go:153] Registering CRI-O factory
Jul 21 12:51:06.512292 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.512244    2563 factory.go:223] Registration of the crio container factory successfully
Jul 21 12:51:06.512356 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.512296    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 21 12:51:06.512356 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.512332    2563 factory.go:103] Registering Raw factory
Jul 21 12:51:06.512356 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.512348    2563 manager.go:1196] Started watching for new ooms in manager
Jul 21 12:51:06.513695 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.513671    2563 manager.go:349] Starting recovery of all containers
Jul 21 12:51:06.513840 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.513815    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bnsst"
Jul 21 12:51:06.515743 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.515716    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-74.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 21 12:51:06.524225 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.524202    2563 manager.go:354] Recovery completed
Jul 21 12:51:06.528372 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.528353    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:51:06.528741 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.528632    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:51:06.530589 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.530573    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:51:06.530665 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.530606    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:51:06.530665 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.530621    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:51:06.531180 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.531164    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 21 12:51:06.531180 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.531180    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 21 12:51:06.531305 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.531198    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 12:51:06.531305 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.531170    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:51:06.531830 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.531816    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:51:06.531888 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.531839    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:51:06.531888 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.531849    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:51:06.533362 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.533350    2563 policy_none.go:49] "None policy: Start"
Jul 21 12:51:06.533412 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.533365    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 21 12:51:06.533412 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.533375    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 21 12:51:06.535417 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.535405    2563 policy_none.go:47] "Start"
Jul 21 12:51:06.569512 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.569496    2563 manager.go:337] "Starting Device Plugin manager"
Jul 21 12:51:06.569598 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.569535    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 21 12:51:06.569598 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.569549    2563 server.go:87] "Starting device plugin registration server"
Jul 21 12:51:06.569817 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.569803    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 21 12:51:06.569877 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.569821    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 21 12:51:06.569926 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.569911    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 21 12:51:06.570056 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.570034    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 21 12:51:06.570056 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.570047    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 21 12:51:06.570600 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.570580    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 21 12:51:06.570668 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.570636    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-74.ec2.internal\" not found"
Jul 21 12:51:06.607901 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.607870    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 21 12:51:06.616417 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.609118    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 21 12:51:06.616417 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.609167    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 21 12:51:06.616417 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.609201    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 21 12:51:06.616417 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.609242    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 21 12:51:06.616417 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.612177    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:51:06.670292 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.670203    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:51:06.671660 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.671640    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:51:06.671743 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.671676    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:51:06.671743 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.671685    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:51:06.671743 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.671707    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.687004 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.686986    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.687059 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.687010    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-74.ec2.internal\": node \"ip-10-0-128-74.ec2.internal\" not found"
Jul 21 12:51:06.709317 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.709289    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-74.ec2.internal"]
Jul 21 12:51:06.709410 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.709351    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:51:06.710785 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.710766    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:51:06.710840 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.710803    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:51:06.710840 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.710817    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:51:06.712996 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.712982    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:51:06.713097 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.713081    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.713140 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.713113    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:51:06.713611 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.713595    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:51:06.713698 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.713628    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:51:06.713698 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.713643    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:51:06.713698 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.713601    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:51:06.713698 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.713700    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:51:06.713870 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.713712    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:51:06.715823 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.715806    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.715895 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.715837    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:51:06.716866 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.716850    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:51:06.716938 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.716878    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:51:06.716938 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.716889    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:51:06.733448 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.733423    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-74.ec2.internal\" not found"
Jul 21 12:51:06.743612 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.743595    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-74.ec2.internal\" not found" node="ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.748092 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.748077    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-74.ec2.internal\" not found" node="ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.814022 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.813997    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b98dd9609e64d70799acdc2e03bf5112-config\") pod \"kube-apiserver-proxy-ip-10-0-128-74.ec2.internal\" (UID: \"b98dd9609e64d70799acdc2e03bf5112\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.814135 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.814029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e50cbf48d80ed7e1e51ef052ff69261d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal\" (UID: \"e50cbf48d80ed7e1e51ef052ff69261d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.814135 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.814047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e50cbf48d80ed7e1e51ef052ff69261d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal\" (UID: \"e50cbf48d80ed7e1e51ef052ff69261d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.834237 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.834212    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-74.ec2.internal\" not found"
Jul 21 12:51:06.914807 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.914775    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e50cbf48d80ed7e1e51ef052ff69261d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal\" (UID: \"e50cbf48d80ed7e1e51ef052ff69261d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.914876 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.914810    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e50cbf48d80ed7e1e51ef052ff69261d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal\" (UID: \"e50cbf48d80ed7e1e51ef052ff69261d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.914876 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.914839    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b98dd9609e64d70799acdc2e03bf5112-config\") pod \"kube-apiserver-proxy-ip-10-0-128-74.ec2.internal\" (UID: \"b98dd9609e64d70799acdc2e03bf5112\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.914932 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.914877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b98dd9609e64d70799acdc2e03bf5112-config\") pod \"kube-apiserver-proxy-ip-10-0-128-74.ec2.internal\" (UID: \"b98dd9609e64d70799acdc2e03bf5112\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.914932 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.914882    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e50cbf48d80ed7e1e51ef052ff69261d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal\" (UID: \"e50cbf48d80ed7e1e51ef052ff69261d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.914932 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:06.914876    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e50cbf48d80ed7e1e51ef052ff69261d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal\" (UID: \"e50cbf48d80ed7e1e51ef052ff69261d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:06.935154 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:06.935082    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-74.ec2.internal\" not found"
Jul 21 12:51:07.036112 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:07.036065    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-74.ec2.internal\" not found"
Jul 21 12:51:07.047259 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:07.047235    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:07.050950 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:07.050936    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:07.136347 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:07.136294    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-74.ec2.internal\" not found"
Jul 21 12:51:07.236915 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:07.236850    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-74.ec2.internal\" not found"
Jul 21 12:51:07.337368 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:07.337334    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-74.ec2.internal\" not found"
Jul 21 12:51:08.818248 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:08.818213    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-74.ec2.internal\" not found"
Jul 21 12:51:08.823355 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:08.823335    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 21 12:51:08.823514 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:08.823497    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 12:51:08.823551 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:08.823500    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 12:51:08.899869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:08.899823    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 12:46:07 +0000 UTC" deadline="2027-12-28 11:29:46.663129187 +0000 UTC"
Jul 21 12:51:08.899869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:08.899854    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12598h38m37.76327941s"
Jul 21 12:51:08.918683 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:08.918656    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-74.ec2.internal\" not found"
Jul 21 12:51:08.943623 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:08.943267    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:51:09.000246 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:09.000214    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb98dd9609e64d70799acdc2e03bf5112.slice/crio-32c2d498f19ec1d2bc977e78007a27308ec59b4d83a45511f1e4d77b38587492 WatchSource:0}: Error finding container 32c2d498f19ec1d2bc977e78007a27308ec59b4d83a45511f1e4d77b38587492: Status 404 returned error can't find the container with id 32c2d498f19ec1d2bc977e78007a27308ec59b4d83a45511f1e4d77b38587492
Jul 21 12:51:09.006051 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.006035    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:51:09.019059 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:09.019029    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-74.ec2.internal\" not found"
Jul 21 12:51:09.025801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.025776    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:51:09.041674 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:09.041647    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50cbf48d80ed7e1e51ef052ff69261d.slice/crio-68b8730b037be87931204ef040f725936ed10745dc9aab7c6ec969472fa15582 WatchSource:0}: Error finding container 68b8730b037be87931204ef040f725936ed10745dc9aab7c6ec969472fa15582: Status 404 returned error can't find the container with id 68b8730b037be87931204ef040f725936ed10745dc9aab7c6ec969472fa15582
Jul 21 12:51:09.091747 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.091709    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:09.104008 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.103987    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 12:51:09.105303 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.105281    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-74.ec2.internal"
Jul 21 12:51:09.113892 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.113869    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 12:51:09.793015 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.792978    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:51:09.870841 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.870803    2563 apiserver.go:52] "Watching apiserver"
Jul 21 12:51:09.879338 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.879279    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 21 12:51:09.880083 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.880008    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-8f5wn","openshift-multus/multus-additional-cni-plugins-nlt9j","openshift-network-diagnostics/network-check-target-kpnng","kube-system/global-pull-secret-syncer-n7pwd","openshift-dns/node-resolver-w92wm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal","openshift-multus/network-metrics-daemon-rjmvf","openshift-network-operator/iptables-alerter-gcqc9","openshift-ovn-kubernetes/ovnkube-node-g9674","kube-system/konnectivity-agent-5w7wb","kube-system/kube-apiserver-proxy-ip-10-0-128-74.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt","openshift-cluster-node-tuning-operator/tuned-lhp7h","openshift-image-registry/node-ca-nln7n"]
Jul 21 12:51:09.882956 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.882935    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:09.883067 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:09.883011    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:09.885361 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.885343    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.887696 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.887678    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:09.887798 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:09.887747    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:09.888025 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.888007    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vq8nz\""
Jul 21 12:51:09.888216 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.888200    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:51:09.888999 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.888978    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 21 12:51:09.890333 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.890315    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:09.890419 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:09.890391    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:09.894472 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.892000    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 21 12:51:09.894472 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.893104    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w92wm"
Jul 21 12:51:09.895419 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.894986    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nln7n"
Jul 21 12:51:09.895582 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.895562    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fkslm\""
Jul 21 12:51:09.895656 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.895568    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 21 12:51:09.895656 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.895604    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 21 12:51:09.898053 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.897271    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 21 12:51:09.898053 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.897310    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lrcls\""
Jul 21 12:51:09.898053 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.897538    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 21 12:51:09.898053 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.897657    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 21 12:51:09.898053 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.897788    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:09.899736 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.899714    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:09.900814 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.900540    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 21 12:51:09.900814 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.900566    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 21 12:51:09.901778 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.901456    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 21 12:51:09.901778 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.901552    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 21 12:51:09.901778 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.901577    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rdfqg\""
Jul 21 12:51:09.901778 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.901552    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 21 12:51:09.902019 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.901938    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 21 12:51:09.902447 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.902238    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 21 12:51:09.904005 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.903967    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 21 12:51:09.904142 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.904124    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 21 12:51:09.904575 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.904551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs\") pod \"network-metrics-daemon-rjmvf\" (UID: \"e4069bb1-2913-45bf-ade6-79ab04c94fa8\") " pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:09.904665 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.904589    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-modprobe-d\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.904665 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.904606    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 21 12:51:09.904665 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.904622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69efe2b6-b508-459f-a900-baec7166c3ed-tmp\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.904814 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.904718    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcebc074-dea4-4fe1-9208-f01443f8720b-tmp-dir\") pod \"node-resolver-w92wm\" (UID: \"fcebc074-dea4-4fe1-9208-f01443f8720b\") " pod="openshift-dns/node-resolver-w92wm"
Jul 21 12:51:09.904814 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.904753    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-sysctl-conf\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.904814 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.904779    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-var-lib-kubelet\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.904814 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.904801    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-njxm7\""
Jul 21 12:51:09.904814 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.904809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-systemd\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.905028 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.904910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-run\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.905028 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.904946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5sxv\" (UniqueName: \"kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv\") pod \"network-check-target-kpnng\" (UID: \"1f77f91a-1189-4c6c-b6cf-d5213e58ec43\") " pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:09.905028 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.904978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/49c51a2b-86c0-4f43-af87-0f9dc3a56986-kubelet-config\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:09.905028 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.905008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fcebc074-dea4-4fe1-9208-f01443f8720b-hosts-file\") pod \"node-resolver-w92wm\" (UID: \"fcebc074-dea4-4fe1-9208-f01443f8720b\") " pod="openshift-dns/node-resolver-w92wm"
Jul 21 12:51:09.905198 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.905102    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-sysconfig\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.905198 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.905172    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hdqj\" (UniqueName: \"kubernetes.io/projected/5bc0b65d-775f-43f5-b762-26ae86e132d1-kube-api-access-9hdqj\") pod \"node-ca-nln7n\" (UID: \"5bc0b65d-775f-43f5-b762-26ae86e132d1\") " pod="openshift-image-registry/node-ca-nln7n"
Jul 21 12:51:09.905284 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.905208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4zdp\" (UniqueName: \"kubernetes.io/projected/e4069bb1-2913-45bf-ade6-79ab04c94fa8-kube-api-access-j4zdp\") pod \"network-metrics-daemon-rjmvf\" (UID: \"e4069bb1-2913-45bf-ade6-79ab04c94fa8\") " pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:09.905373 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.905351    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-lib-modules\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.905516 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.905498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/69efe2b6-b508-459f-a900-baec7166c3ed-etc-tuned\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.905698 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.905558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-995df\" (UniqueName: \"kubernetes.io/projected/69efe2b6-b508-459f-a900-baec7166c3ed-kube-api-access-995df\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.905775 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.905753    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:09.905806 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.905794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:09.905836 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.905817    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bc0b65d-775f-43f5-b762-26ae86e132d1-host\") pod \"node-ca-nln7n\" (UID: \"5bc0b65d-775f-43f5-b762-26ae86e132d1\") " pod="openshift-image-registry/node-ca-nln7n"
Jul 21 12:51:09.905984 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.905972    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-kubernetes\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.906018 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.905995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-sys\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.906018 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.906012    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-host\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.906072 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.906030    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnfck\" (UniqueName: \"kubernetes.io/projected/fcebc074-dea4-4fe1-9208-f01443f8720b-kube-api-access-lnfck\") pod \"node-resolver-w92wm\" (UID: \"fcebc074-dea4-4fe1-9208-f01443f8720b\") " pod="openshift-dns/node-resolver-w92wm"
Jul 21 12:51:09.906451 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.906437    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5bc0b65d-775f-43f5-b762-26ae86e132d1-serviceca\") pod \"node-ca-nln7n\" (UID: \"5bc0b65d-775f-43f5-b762-26ae86e132d1\") " pod="openshift-image-registry/node-ca-nln7n"
Jul 21 12:51:09.906491 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.906465    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-sysctl-d\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:09.906491 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.906484    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/49c51a2b-86c0-4f43-af87-0f9dc3a56986-dbus\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:09.908398 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.908378    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 12:51:09.908724 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.908707    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 21 12:51:09.908913 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.908898    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-45mz8\""
Jul 21 12:51:09.909104 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.909090    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 21 12:51:09.909303 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.909276    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5w7wb"
Jul 21 12:51:09.911084 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.910767    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gcqc9"
Jul 21 12:51:09.911588 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.911567    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 21 12:51:09.911723 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.911709    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mp7dp\""
Jul 21 12:51:09.911889 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.911871    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 21 12:51:09.914079 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.912873    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:51:09.914079 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.913175    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:09.914079 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.913481    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 21 12:51:09.914079 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.913534    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 21 12:51:09.914079 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.913991    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7gqzh\""
Jul 21 12:51:09.915765 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.915745    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 21 12:51:09.915977 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.915963    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 21 12:51:09.916054 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.916025    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wg5np\""
Jul 21 12:51:09.916110 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.916073    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 21 12:51:09.930002 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.929985    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-28p8m"
Jul 21 12:51:09.938138 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.938117    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-28p8m"
Jul 21 12:51:09.993019 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.992991    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 21 12:51:09.994840 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.994784    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal" event={"ID":"e50cbf48d80ed7e1e51ef052ff69261d","Type":"ContainerStarted","Data":"68b8730b037be87931204ef040f725936ed10745dc9aab7c6ec969472fa15582"}
Jul 21 12:51:09.995977 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:09.995955    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-74.ec2.internal" event={"ID":"b98dd9609e64d70799acdc2e03bf5112","Type":"ContainerStarted","Data":"32c2d498f19ec1d2bc977e78007a27308ec59b4d83a45511f1e4d77b38587492"}
Jul 21 12:51:10.006787 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.006766    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-run-netns\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.006912 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.006798    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b6b9e745-a2a2-4380-8f24-270804bc742f-multus-daemon-config\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.006912 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.006822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.006912 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.006854    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-run-ovn\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.006912 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.006874    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-ovnkube-script-lib\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.006912 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.006891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.007128 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.006926    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/49c51a2b-86c0-4f43-af87-0f9dc3a56986-kubelet-config\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:10.007128 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.006944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-sysconfig\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.007128 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.006960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-run-k8s-cni-cncf-io\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.007128 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.006979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzkn6\" (UniqueName: \"kubernetes.io/projected/b6b9e745-a2a2-4380-8f24-270804bc742f-kube-api-access-tzkn6\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.007128 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-slash\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.007128 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007039    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-ovn-node-metrics-cert\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.007128 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007045    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-sysconfig\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.007128 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4zdp\" (UniqueName: \"kubernetes.io/projected/e4069bb1-2913-45bf-ade6-79ab04c94fa8-kube-api-access-j4zdp\") pod \"network-metrics-daemon-rjmvf\" (UID: \"e4069bb1-2913-45bf-ade6-79ab04c94fa8\") " pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:10.007128 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007119    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-lib-modules\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.007537 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007056    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/49c51a2b-86c0-4f43-af87-0f9dc3a56986-kubelet-config\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:10.007537 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007154    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-multus-socket-dir-parent\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.007537 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007217    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47-agent-certs\") pod \"konnectivity-agent-5w7wb\" (UID: \"50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47\") " pod="kube-system/konnectivity-agent-5w7wb"
Jul 21 12:51:10.007679 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007577    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-lib-modules\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.007801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007256    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:10.007869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007830    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-kubernetes\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.007869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-system-cni-dir\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.007957 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm2m5\" (UniqueName: \"kubernetes.io/projected/ea43418c-d593-4ed0-b999-a1329a5caab3-kube-api-access-zm2m5\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.007957 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-node-log\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.007957 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.008101 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnfck\" (UniqueName: \"kubernetes.io/projected/fcebc074-dea4-4fe1-9208-f01443f8720b-kube-api-access-lnfck\") pod \"node-resolver-w92wm\" (UID: \"fcebc074-dea4-4fe1-9208-f01443f8720b\") " pod="openshift-dns/node-resolver-w92wm"
Jul 21 12:51:10.008101 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.007981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5bc0b65d-775f-43f5-b762-26ae86e132d1-serviceca\") pod \"node-ca-nln7n\" (UID: \"5bc0b65d-775f-43f5-b762-26ae86e132d1\") " pod="openshift-image-registry/node-ca-nln7n"
Jul 21 12:51:10.008101 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.008000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-run-multus-certs\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.008101 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.008016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-socket-dir\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.008101 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.008034    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-run-openvswitch\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.008101 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.008053    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/49c51a2b-86c0-4f43-af87-0f9dc3a56986-dbus\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:10.008392 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.008191    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:10.008392 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.008309    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret podName:49c51a2b-86c0-4f43-af87-0f9dc3a56986 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:10.508261118 +0000 UTC m=+3.033764269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret") pod "global-pull-secret-syncer-n7pwd" (UID: "49c51a2b-86c0-4f43-af87-0f9dc3a56986") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:10.008721 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.008522    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-kubernetes\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.008721 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.008685    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/49c51a2b-86c0-4f43-af87-0f9dc3a56986-dbus\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:10.008967 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.008743    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5bc0b65d-775f-43f5-b762-26ae86e132d1-serviceca\") pod \"node-ca-nln7n\" (UID: \"5bc0b65d-775f-43f5-b762-26ae86e132d1\") " pod="openshift-image-registry/node-ca-nln7n"
Jul 21 12:51:10.008967 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.008869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-kubelet\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.008967 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.008910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-cni-bin\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.008967 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.008927    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-env-overrides\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.008967 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.008960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs\") pod \"network-metrics-daemon-rjmvf\" (UID: \"e4069bb1-2913-45bf-ade6-79ab04c94fa8\") " pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:10.009185 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.008986    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-modprobe-d\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.009185 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-var-lib-cni-multus\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.009185 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009048    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67z26\" (UniqueName: \"kubernetes.io/projected/2b2479d2-3197-427c-9dc0-da1b80bb6826-kube-api-access-67z26\") pod \"iptables-alerter-gcqc9\" (UID: \"2b2479d2-3197-427c-9dc0-da1b80bb6826\") " pod="openshift-network-operator/iptables-alerter-gcqc9"
Jul 21 12:51:10.009185 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009069    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-var-lib-openvswitch\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.009185 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.009185 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009124    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-os-release\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.009185 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-cni-netd\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.009185 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-ovnkube-config\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009188    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52f8j\" (UniqueName: \"kubernetes.io/projected/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-kube-api-access-52f8j\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009230    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5sxv\" (UniqueName: \"kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv\") pod \"network-check-target-kpnng\" (UID: \"1f77f91a-1189-4c6c-b6cf-d5213e58ec43\") " pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-var-lib-cni-bin\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009264    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b2479d2-3197-427c-9dc0-da1b80bb6826-iptables-alerter-script\") pod \"iptables-alerter-gcqc9\" (UID: \"2b2479d2-3197-427c-9dc0-da1b80bb6826\") " pod="openshift-network-operator/iptables-alerter-gcqc9"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009282    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-run-systemd\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-etc-openvswitch\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fcebc074-dea4-4fe1-9208-f01443f8720b-hosts-file\") pod \"node-resolver-w92wm\" (UID: \"fcebc074-dea4-4fe1-9208-f01443f8720b\") " pod="openshift-dns/node-resolver-w92wm"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-cnibin\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009408    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-run-netns\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009436    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-run-ovn-kubernetes\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-system-cni-dir\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009490    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-modprobe-d\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-os-release\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.009527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009525    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47-konnectivity-ca\") pod \"konnectivity-agent-5w7wb\" (UID: \"50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47\") " pod="kube-system/konnectivity-agent-5w7wb"
Jul 21 12:51:10.010126 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009580    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hdqj\" (UniqueName: \"kubernetes.io/projected/5bc0b65d-775f-43f5-b762-26ae86e132d1-kube-api-access-9hdqj\") pod \"node-ca-nln7n\" (UID: \"5bc0b65d-775f-43f5-b762-26ae86e132d1\") " pod="openshift-image-registry/node-ca-nln7n"
Jul 21 12:51:10.010126 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/69efe2b6-b508-459f-a900-baec7166c3ed-etc-tuned\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.010126 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.009691    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:10.010126 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009854    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fcebc074-dea4-4fe1-9208-f01443f8720b-hosts-file\") pod \"node-resolver-w92wm\" (UID: \"fcebc074-dea4-4fe1-9208-f01443f8720b\") " pod="openshift-dns/node-resolver-w92wm"
Jul 21 12:51:10.010126 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.009905    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-995df\" (UniqueName: \"kubernetes.io/projected/69efe2b6-b508-459f-a900-baec7166c3ed-kube-api-access-995df\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.010361 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.010179    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs podName:e4069bb1-2913-45bf-ade6-79ab04c94fa8 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:10.510161363 +0000 UTC m=+3.035664515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs") pod "network-metrics-daemon-rjmvf" (UID: "e4069bb1-2913-45bf-ade6-79ab04c94fa8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:10.010361 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.010187    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.010672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6b9e745-a2a2-4380-8f24-270804bc742f-cni-binary-copy\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.010732    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-sys-fs\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.010777    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bc0b65d-775f-43f5-b762-26ae86e132d1-host\") pod \"node-ca-nln7n\" (UID: \"5bc0b65d-775f-43f5-b762-26ae86e132d1\") " pod="openshift-image-registry/node-ca-nln7n"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.010817    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-sys\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.010830    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bc0b65d-775f-43f5-b762-26ae86e132d1-host\") pod \"node-ca-nln7n\" (UID: \"5bc0b65d-775f-43f5-b762-26ae86e132d1\") " pod="openshift-image-registry/node-ca-nln7n"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.010846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-host\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.010901    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-multus-cni-dir\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.010911    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-sys\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.010933    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-systemd-units\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.010964    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-cnibin\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.010997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-sysctl-d\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2z5z\" (UniqueName: \"kubernetes.io/projected/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-kube-api-access-q2z5z\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b2479d2-3197-427c-9dc0-da1b80bb6826-host-slash\") pod \"iptables-alerter-gcqc9\" (UID: \"2b2479d2-3197-427c-9dc0-da1b80bb6826\") " pod="openshift-network-operator/iptables-alerter-gcqc9"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-cni-binary-copy\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-sysctl-d\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69efe2b6-b508-459f-a900-baec7166c3ed-tmp\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.010965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-host\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011136    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-multus-conf-dir\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011183    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-etc-selinux\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011216    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-log-socket\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011245    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-device-dir\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcebc074-dea4-4fe1-9208-f01443f8720b-tmp-dir\") pod \"node-resolver-w92wm\" (UID: \"fcebc074-dea4-4fe1-9208-f01443f8720b\") " pod="openshift-dns/node-resolver-w92wm"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-sysctl-conf\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-var-lib-kubelet\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-var-lib-kubelet\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011439    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-hostroot\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-etc-kubernetes\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011493    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-registration-dir\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-systemd\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-run\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011682    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-run\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011761    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-var-lib-kubelet\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011826    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-systemd\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.012986 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.011877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/69efe2b6-b508-459f-a900-baec7166c3ed-etc-sysctl-conf\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.013660 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.012090    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcebc074-dea4-4fe1-9208-f01443f8720b-tmp-dir\") pod \"node-resolver-w92wm\" (UID: \"fcebc074-dea4-4fe1-9208-f01443f8720b\") " pod="openshift-dns/node-resolver-w92wm"
Jul 21 12:51:10.013728 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.013708    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/69efe2b6-b508-459f-a900-baec7166c3ed-etc-tuned\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.015328 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.015306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69efe2b6-b508-459f-a900-baec7166c3ed-tmp\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.015882 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.015860    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:51:10.015958 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.015890    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:51:10.015958 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.015904    2563 projected.go:196] Error preparing data for projected volume kube-api-access-m5sxv for pod openshift-network-diagnostics/network-check-target-kpnng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:10.016109 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.015963    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv podName:1f77f91a-1189-4c6c-b6cf-d5213e58ec43 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:10.515948606 +0000 UTC m=+3.041451757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m5sxv" (UniqueName: "kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv") pod "network-check-target-kpnng" (UID: "1f77f91a-1189-4c6c-b6cf-d5213e58ec43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:10.017284 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.017263    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4zdp\" (UniqueName: \"kubernetes.io/projected/e4069bb1-2913-45bf-ade6-79ab04c94fa8-kube-api-access-j4zdp\") pod \"network-metrics-daemon-rjmvf\" (UID: \"e4069bb1-2913-45bf-ade6-79ab04c94fa8\") " pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:10.017647 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.017629    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnfck\" (UniqueName: \"kubernetes.io/projected/fcebc074-dea4-4fe1-9208-f01443f8720b-kube-api-access-lnfck\") pod \"node-resolver-w92wm\" (UID: \"fcebc074-dea4-4fe1-9208-f01443f8720b\") " pod="openshift-dns/node-resolver-w92wm"
Jul 21 12:51:10.021380 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.021358    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hdqj\" (UniqueName: \"kubernetes.io/projected/5bc0b65d-775f-43f5-b762-26ae86e132d1-kube-api-access-9hdqj\") pod \"node-ca-nln7n\" (UID: \"5bc0b65d-775f-43f5-b762-26ae86e132d1\") " pod="openshift-image-registry/node-ca-nln7n"
Jul 21 12:51:10.025013 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.024994    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-995df\" (UniqueName: \"kubernetes.io/projected/69efe2b6-b508-459f-a900-baec7166c3ed-kube-api-access-995df\") pod \"tuned-lhp7h\" (UID: \"69efe2b6-b508-459f-a900-baec7166c3ed\") " pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.057637 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.057554    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:51:10.112711 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-var-lib-cni-bin\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.112711 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b2479d2-3197-427c-9dc0-da1b80bb6826-iptables-alerter-script\") pod \"iptables-alerter-gcqc9\" (UID: \"2b2479d2-3197-427c-9dc0-da1b80bb6826\") " pod="openshift-network-operator/iptables-alerter-gcqc9"
Jul 21 12:51:10.112941 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-run-systemd\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.112941 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-etc-openvswitch\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.112941 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-cnibin\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.112941 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-run-netns\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.112941 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112812    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-var-lib-cni-bin\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.112941 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112825    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-run-ovn-kubernetes\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.112941 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112863    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-run-ovn-kubernetes\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.112941 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-system-cni-dir\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.112941 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-os-release\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.112941 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112915    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47-konnectivity-ca\") pod \"konnectivity-agent-5w7wb\" (UID: \"50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47\") " pod="kube-system/konnectivity-agent-5w7wb"
Jul 21 12:51:10.112941 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112925    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-system-cni-dir\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112957    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-run-systemd\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6b9e745-a2a2-4380-8f24-270804bc742f-cni-binary-copy\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.112987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-sys-fs\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113004    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-cnibin\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-multus-cni-dir\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113054    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-run-netns\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-systemd-units\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113085    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-cnibin\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113113    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2z5z\" (UniqueName: \"kubernetes.io/projected/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-kube-api-access-q2z5z\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b2479d2-3197-427c-9dc0-da1b80bb6826-host-slash\") pod \"iptables-alerter-gcqc9\" (UID: \"2b2479d2-3197-427c-9dc0-da1b80bb6826\") " pod="openshift-network-operator/iptables-alerter-gcqc9"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113142    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-os-release\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113154    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-cni-binary-copy\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-multus-conf-dir\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113187    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-sys-fs\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-etc-selinux\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-etc-openvswitch\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.113498 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113362    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b2479d2-3197-427c-9dc0-da1b80bb6826-iptables-alerter-script\") pod \"iptables-alerter-gcqc9\" (UID: \"2b2479d2-3197-427c-9dc0-da1b80bb6826\") " pod="openshift-network-operator/iptables-alerter-gcqc9"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113424    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-cnibin\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113794    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6b9e745-a2a2-4380-8f24-270804bc742f-cni-binary-copy\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113815    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-cni-binary-copy\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-log-socket\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113866    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-device-dir\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113876    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-systemd-units\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-var-lib-kubelet\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113928    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b2479d2-3197-427c-9dc0-da1b80bb6826-host-slash\") pod \"iptables-alerter-gcqc9\" (UID: \"2b2479d2-3197-427c-9dc0-da1b80bb6826\") " pod="openshift-network-operator/iptables-alerter-gcqc9"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113938    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47-konnectivity-ca\") pod \"konnectivity-agent-5w7wb\" (UID: \"50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47\") " pod="kube-system/konnectivity-agent-5w7wb"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113948    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-multus-cni-dir\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113977    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-var-lib-kubelet\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.113980    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-hostroot\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-etc-kubernetes\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114005    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-log-socket\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114040    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-device-dir\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114054    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-etc-selinux\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-registration-dir\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.114278 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114090    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-hostroot\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114122    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-multus-conf-dir\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-run-netns\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114168    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-registration-dir\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b6b9e745-a2a2-4380-8f24-270804bc742f-multus-daemon-config\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114202    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114287    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-etc-kubernetes\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114358    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-run-netns\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-run-ovn\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-ovnkube-script-lib\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114489    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114521    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-run-k8s-cni-cncf-io\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzkn6\" (UniqueName: \"kubernetes.io/projected/b6b9e745-a2a2-4380-8f24-270804bc742f-kube-api-access-tzkn6\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114586    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-slash\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-ovn-node-metrics-cert\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114639    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b6b9e745-a2a2-4380-8f24-270804bc742f-multus-daemon-config\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114654    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-run-k8s-cni-cncf-io\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114695    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.115869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.114915    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-run-ovn\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.115869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115375    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.115869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115393    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-slash\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.115869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-multus-socket-dir-parent\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47-agent-certs\") pod \"konnectivity-agent-5w7wb\" (UID: \"50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47\") " pod="kube-system/konnectivity-agent-5w7wb"
Jul 21 12:51:10.115869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115540    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-multus-socket-dir-parent\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115709    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-system-cni-dir\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-ovnkube-script-lib\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.115869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115760    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-system-cni-dir\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.115869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm2m5\" (UniqueName: \"kubernetes.io/projected/ea43418c-d593-4ed0-b999-a1329a5caab3-kube-api-access-zm2m5\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.115869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-node-log\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.115869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115875    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-run-multus-certs\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-node-log\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115926    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-socket-dir\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-run-openvswitch\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.115955    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-run-multus-certs\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-kubelet\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116038    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-cni-bin\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116045    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea43418c-d593-4ed0-b999-a1329a5caab3-socket-dir\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116053    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-env-overrides\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-kubelet\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116115    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-cni-bin\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-var-lib-cni-multus\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67z26\" (UniqueName: \"kubernetes.io/projected/2b2479d2-3197-427c-9dc0-da1b80bb6826-kube-api-access-67z26\") pod \"iptables-alerter-gcqc9\" (UID: \"2b2479d2-3197-427c-9dc0-da1b80bb6826\") " pod="openshift-network-operator/iptables-alerter-gcqc9"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-run-openvswitch\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-var-lib-openvswitch\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.116655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116207    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-host-var-lib-cni-multus\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.117547 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.117547 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116277    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-os-release\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.117547 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-cni-netd\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.117547 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116355    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-var-lib-openvswitch\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.117547 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-ovnkube-config\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.117547 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116392    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-env-overrides\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.117547 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116403    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52f8j\" (UniqueName: \"kubernetes.io/projected/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-kube-api-access-52f8j\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.117547 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116409    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b6b9e745-a2a2-4380-8f24-270804bc742f-os-release\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.117547 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.117547 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.117547 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116510    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-host-cni-netd\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.117547 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.116818    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.118047 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.117553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-ovnkube-config\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.118047 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.117940    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-ovn-node-metrics-cert\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.118283 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.118265    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47-agent-certs\") pod \"konnectivity-agent-5w7wb\" (UID: \"50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47\") " pod="kube-system/konnectivity-agent-5w7wb"
Jul 21 12:51:10.128803 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.128778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzkn6\" (UniqueName: \"kubernetes.io/projected/b6b9e745-a2a2-4380-8f24-270804bc742f-kube-api-access-tzkn6\") pod \"multus-8f5wn\" (UID: \"b6b9e745-a2a2-4380-8f24-270804bc742f\") " pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.128928 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.128839    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2z5z\" (UniqueName: \"kubernetes.io/projected/b1e589a5-672d-48e9-b4dc-aa3cbf2e551d-kube-api-access-q2z5z\") pod \"multus-additional-cni-plugins-nlt9j\" (UID: \"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d\") " pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.129731 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.129706    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67z26\" (UniqueName: \"kubernetes.io/projected/2b2479d2-3197-427c-9dc0-da1b80bb6826-kube-api-access-67z26\") pod \"iptables-alerter-gcqc9\" (UID: \"2b2479d2-3197-427c-9dc0-da1b80bb6826\") " pod="openshift-network-operator/iptables-alerter-gcqc9"
Jul 21 12:51:10.129846 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.129756    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52f8j\" (UniqueName: \"kubernetes.io/projected/cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b-kube-api-access-52f8j\") pod \"ovnkube-node-g9674\" (UID: \"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.131133 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.131110    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm2m5\" (UniqueName: \"kubernetes.io/projected/ea43418c-d593-4ed0-b999-a1329a5caab3-kube-api-access-zm2m5\") pod \"aws-ebs-csi-driver-node-7dzgt\" (UID: \"ea43418c-d593-4ed0-b999-a1329a5caab3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.197211 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.197173    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lhp7h"
Jul 21 12:51:10.210815 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.210790    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w92wm"
Jul 21 12:51:10.219511 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.219488    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nln7n"
Jul 21 12:51:10.225309 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.225277    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:10.231923 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.231905    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8f5wn"
Jul 21 12:51:10.237632 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.237602    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nlt9j"
Jul 21 12:51:10.244236 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.244217    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5w7wb"
Jul 21 12:51:10.250906 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.250885    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gcqc9"
Jul 21 12:51:10.256598 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.256580    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt"
Jul 21 12:51:10.520389 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.520319    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs\") pod \"network-metrics-daemon-rjmvf\" (UID: \"e4069bb1-2913-45bf-ade6-79ab04c94fa8\") " pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:10.520389 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.520378    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5sxv\" (UniqueName: \"kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv\") pod \"network-check-target-kpnng\" (UID: \"1f77f91a-1189-4c6c-b6cf-d5213e58ec43\") " pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:10.520563 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.520466    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:10.520563 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.520477    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:10.520563 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.520527    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs podName:e4069bb1-2913-45bf-ade6-79ab04c94fa8 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:11.520512487 +0000 UTC m=+4.046015622 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs") pod "network-metrics-daemon-rjmvf" (UID: "e4069bb1-2913-45bf-ade6-79ab04c94fa8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:10.520563 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.520553    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:51:10.520715 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.520578    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:51:10.520715 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.520583    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:10.520715 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.520591    2563 projected.go:196] Error preparing data for projected volume kube-api-access-m5sxv for pod openshift-network-diagnostics/network-check-target-kpnng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:10.520715 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.520640    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret podName:49c51a2b-86c0-4f43-af87-0f9dc3a56986 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:11.520623892 +0000 UTC m=+4.046127047 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret") pod "global-pull-secret-syncer-n7pwd" (UID: "49c51a2b-86c0-4f43-af87-0f9dc3a56986") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:10.520715 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:10.520664    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv podName:1f77f91a-1189-4c6c-b6cf-d5213e58ec43 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:11.520654892 +0000 UTC m=+4.046158029 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5sxv" (UniqueName: "kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv") pod "network-check-target-kpnng" (UID: "1f77f91a-1189-4c6c-b6cf-d5213e58ec43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:10.899037 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:10.898930    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50c5c92e_e9f6_4856_bc1d_4a6ef81c7f47.slice/crio-a97840bcb367ceb45df5293d4dd4760845da1ded2561e39aeb686fd5116f1774 WatchSource:0}: Error finding container a97840bcb367ceb45df5293d4dd4760845da1ded2561e39aeb686fd5116f1774: Status 404 returned error can't find the container with id a97840bcb367ceb45df5293d4dd4760845da1ded2561e39aeb686fd5116f1774
Jul 21 12:51:10.941413 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.941139    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:46:09 +0000 UTC" deadline="2027-12-20 09:44:51.119049056 +0000 UTC"
Jul 21 12:51:10.941508 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:10.941415    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12404h53m40.177641503s"
Jul 21 12:51:11.002849 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.002808    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-74.ec2.internal" event={"ID":"b98dd9609e64d70799acdc2e03bf5112","Type":"ContainerStarted","Data":"8be0d259d0620ca7bd2df3c5642427caba89387a3ae65241b14da34697a358cf"}
Jul 21 12:51:11.006353 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.006320    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nlt9j" event={"ID":"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d","Type":"ContainerStarted","Data":"a130a1aca437505a4d87bbe8432eaf55a3d0931b4b8539d848c89be835e345ea"}
Jul 21 12:51:11.007427 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.007398    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nln7n" event={"ID":"5bc0b65d-775f-43f5-b762-26ae86e132d1","Type":"ContainerStarted","Data":"c405854ccc3778e57ea96fdf04db1ad4ba00f4d0388e479216e63a271de39fac"}
Jul 21 12:51:11.008730 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.008703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lhp7h" event={"ID":"69efe2b6-b508-459f-a900-baec7166c3ed","Type":"ContainerStarted","Data":"8543283fe47d367ba008f67bd2b151bf12d36c412307b913d4f330cdcb56a866"}
Jul 21 12:51:11.009762 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.009738    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gcqc9" event={"ID":"2b2479d2-3197-427c-9dc0-da1b80bb6826","Type":"ContainerStarted","Data":"ddf6243641f43a5bd28a395698d0e4aac0c98f5fcf428083912afb2a2dc027fa"}
Jul 21 12:51:11.011762 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.011740    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9674" event={"ID":"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b","Type":"ContainerStarted","Data":"00d1c63d294f66a9efc4c94344d87d1e63f53cf85bd866746252963303ad6c56"}
Jul 21 12:51:11.014761 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.014740    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5w7wb" event={"ID":"50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47","Type":"ContainerStarted","Data":"a97840bcb367ceb45df5293d4dd4760845da1ded2561e39aeb686fd5116f1774"}
Jul 21 12:51:11.017416 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.017396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt" event={"ID":"ea43418c-d593-4ed0-b999-a1329a5caab3","Type":"ContainerStarted","Data":"1ba0b60444786829d818226e1611e7fdefdc81bf7fabc2084a4b1771a9515e2a"}
Jul 21 12:51:11.018450 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.018430    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8f5wn" event={"ID":"b6b9e745-a2a2-4380-8f24-270804bc742f","Type":"ContainerStarted","Data":"da46f74668c645a01f92cbe57a94b45dfce3cbdceac9df40f6971bf60333e900"}
Jul 21 12:51:11.018889 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.018849    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-74.ec2.internal" podStartSLOduration=2.018838733 podStartE2EDuration="2.018838733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:11.01861479 +0000 UTC m=+3.544117944" watchObservedRunningTime="2026-07-21 12:51:11.018838733 +0000 UTC m=+3.544341887"
Jul 21 12:51:11.020473 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.020455    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w92wm" event={"ID":"fcebc074-dea4-4fe1-9208-f01443f8720b","Type":"ContainerStarted","Data":"982c2ce4300e596eaccc2a31e935a08ed77d9f6e51b1b0faf291237ad7e68012"}
Jul 21 12:51:11.528586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.527508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5sxv\" (UniqueName: \"kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv\") pod \"network-check-target-kpnng\" (UID: \"1f77f91a-1189-4c6c-b6cf-d5213e58ec43\") " pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:11.528586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.527620    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:11.528586 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.527657    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs\") pod \"network-metrics-daemon-rjmvf\" (UID: \"e4069bb1-2913-45bf-ade6-79ab04c94fa8\") " pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:11.528586 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:11.527783    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:11.528586 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:11.527846    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs podName:e4069bb1-2913-45bf-ade6-79ab04c94fa8 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:13.527827319 +0000 UTC m=+6.053330475 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs") pod "network-metrics-daemon-rjmvf" (UID: "e4069bb1-2913-45bf-ade6-79ab04c94fa8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:11.528586 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:11.528326    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:51:11.528586 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:11.528349    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:51:11.528586 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:11.528362    2563 projected.go:196] Error preparing data for projected volume kube-api-access-m5sxv for pod openshift-network-diagnostics/network-check-target-kpnng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:11.528586 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:11.528410    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv podName:1f77f91a-1189-4c6c-b6cf-d5213e58ec43 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:13.528395069 +0000 UTC m=+6.053898205 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5sxv" (UniqueName: "kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv") pod "network-check-target-kpnng" (UID: "1f77f91a-1189-4c6c-b6cf-d5213e58ec43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:11.528586 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:11.528471    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:11.528586 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:11.528505    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret podName:49c51a2b-86c0-4f43-af87-0f9dc3a56986 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:13.528494172 +0000 UTC m=+6.053997306 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret") pod "global-pull-secret-syncer-n7pwd" (UID: "49c51a2b-86c0-4f43-af87-0f9dc3a56986") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:11.942522 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.942367    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:46:09 +0000 UTC" deadline="2028-01-23 00:50:08.874485269 +0000 UTC"
Jul 21 12:51:11.942522 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.942406    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13211h58m56.932084667s"
Jul 21 12:51:11.993522 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.992729    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:11.993522 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:11.992865    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:11.993522 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.993250    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:11.993522 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:11.993352    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:11.993522 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:11.993414    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:11.993522 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:11.993485    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:12.041949 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:12.041646    2563 generic.go:356] "Generic (PLEG): container finished" podID="e50cbf48d80ed7e1e51ef052ff69261d" containerID="bbd04b861382af5fcc967f8854078965c950bfd243f102c1df7157a8027e7a08" exitCode=0
Jul 21 12:51:12.041949 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:12.041750    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal" event={"ID":"e50cbf48d80ed7e1e51ef052ff69261d","Type":"ContainerDied","Data":"bbd04b861382af5fcc967f8854078965c950bfd243f102c1df7157a8027e7a08"}
Jul 21 12:51:13.059315 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:13.055746    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal" event={"ID":"e50cbf48d80ed7e1e51ef052ff69261d","Type":"ContainerStarted","Data":"3c93b0dad49cc1fedd1a76af9490749b4833bd0e87c6fc7c9860a04357ab8071"}
Jul 21 12:51:13.071971 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:13.071916    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-74.ec2.internal" podStartSLOduration=4.071894179 podStartE2EDuration="4.071894179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:13.071072153 +0000 UTC m=+5.596575311" watchObservedRunningTime="2026-07-21 12:51:13.071894179 +0000 UTC m=+5.597397333"
Jul 21 12:51:13.546361 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:13.543555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs\") pod \"network-metrics-daemon-rjmvf\" (UID: \"e4069bb1-2913-45bf-ade6-79ab04c94fa8\") " pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:13.546361 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:13.543616    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5sxv\" (UniqueName: \"kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv\") pod \"network-check-target-kpnng\" (UID: \"1f77f91a-1189-4c6c-b6cf-d5213e58ec43\") " pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:13.546361 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:13.543693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:13.550488 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:13.550454    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:13.550642 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:13.550546    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs podName:e4069bb1-2913-45bf-ade6-79ab04c94fa8 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:17.550529432 +0000 UTC m=+10.076032565 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs") pod "network-metrics-daemon-rjmvf" (UID: "e4069bb1-2913-45bf-ade6-79ab04c94fa8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:13.551116 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:13.551078    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:51:13.551116 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:13.551111    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:51:13.551312 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:13.551124    2563 projected.go:196] Error preparing data for projected volume kube-api-access-m5sxv for pod openshift-network-diagnostics/network-check-target-kpnng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:13.551312 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:13.551183    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv podName:1f77f91a-1189-4c6c-b6cf-d5213e58ec43 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:17.551166389 +0000 UTC m=+10.076669523 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5sxv" (UniqueName: "kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv") pod "network-check-target-kpnng" (UID: "1f77f91a-1189-4c6c-b6cf-d5213e58ec43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:13.551423 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:13.551401    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:13.551480 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:13.551464    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret podName:49c51a2b-86c0-4f43-af87-0f9dc3a56986 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:17.551442349 +0000 UTC m=+10.076945485 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret") pod "global-pull-secret-syncer-n7pwd" (UID: "49c51a2b-86c0-4f43-af87-0f9dc3a56986") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:13.990555 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:13.990426    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:13.990716 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:13.990546    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:13.990925 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:13.990903    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:13.991055 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:13.990997    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:13.991129 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:13.991061    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:13.991178 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:13.991128    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:15.991415 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:15.991380    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:15.991788 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:15.991511    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:15.991927 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:15.991893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:15.992039 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:15.992006    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:15.992176 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:15.992148    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:15.992278 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:15.992245    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:17.581529 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:17.581492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:17.581983 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:17.581548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs\") pod \"network-metrics-daemon-rjmvf\" (UID: \"e4069bb1-2913-45bf-ade6-79ab04c94fa8\") " pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:17.581983 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:17.581586    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5sxv\" (UniqueName: \"kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv\") pod \"network-check-target-kpnng\" (UID: \"1f77f91a-1189-4c6c-b6cf-d5213e58ec43\") " pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:17.581983 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:17.581679    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:17.581983 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:17.581727    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:51:17.581983 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:17.581743    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:51:17.581983 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:17.581754    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret podName:49c51a2b-86c0-4f43-af87-0f9dc3a56986 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:25.58173393 +0000 UTC m=+18.107237075 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret") pod "global-pull-secret-syncer-n7pwd" (UID: "49c51a2b-86c0-4f43-af87-0f9dc3a56986") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:17.581983 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:17.581756    2563 projected.go:196] Error preparing data for projected volume kube-api-access-m5sxv for pod openshift-network-diagnostics/network-check-target-kpnng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:17.581983 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:17.581802    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv podName:1f77f91a-1189-4c6c-b6cf-d5213e58ec43 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:25.581791419 +0000 UTC m=+18.107294571 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5sxv" (UniqueName: "kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv") pod "network-check-target-kpnng" (UID: "1f77f91a-1189-4c6c-b6cf-d5213e58ec43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:17.581983 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:17.581812    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:17.581983 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:17.581859    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs podName:e4069bb1-2913-45bf-ade6-79ab04c94fa8 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:25.581844296 +0000 UTC m=+18.107347431 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs") pod "network-metrics-daemon-rjmvf" (UID: "e4069bb1-2913-45bf-ade6-79ab04c94fa8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:17.993132 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:17.990799    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:17.993132 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:17.990910    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:17.993132 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:17.991320    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:17.993132 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:17.991401    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:17.993132 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:17.991446    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:17.993132 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:17.991515    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:19.990366 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:19.990330    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:19.990366 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:19.990356    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:19.990863 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:19.990314    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:19.990863 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:19.990476    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:19.990863 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:19.990619    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:19.991093 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:19.991046    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:21.990340 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:21.990285    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:21.990806 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:21.990285    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:21.990806 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:21.990443    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:21.990806 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:21.990537    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:21.990806 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:21.990285    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:21.990806 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:21.990635    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:23.990460 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:23.990428    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:23.990923 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:23.990441    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:23.990923 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:23.990565    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:23.990923 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:23.990445    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:23.990923 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:23.990635    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:23.990923 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:23.990728    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:25.648021 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:25.647975    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:25.648583 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:25.648037    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs\") pod \"network-metrics-daemon-rjmvf\" (UID: \"e4069bb1-2913-45bf-ade6-79ab04c94fa8\") " pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:25.648583 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:25.648071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5sxv\" (UniqueName: \"kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv\") pod \"network-check-target-kpnng\" (UID: \"1f77f91a-1189-4c6c-b6cf-d5213e58ec43\") " pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:25.648583 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:25.648159    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:25.648583 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:25.648208    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:51:25.648583 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:25.648226    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:51:25.648583 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:25.648238    2563 projected.go:196] Error preparing data for projected volume kube-api-access-m5sxv for pod openshift-network-diagnostics/network-check-target-kpnng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:25.648583 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:25.648158    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:25.648583 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:25.648239    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret podName:49c51a2b-86c0-4f43-af87-0f9dc3a56986 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:41.648218008 +0000 UTC m=+34.173721157 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret") pod "global-pull-secret-syncer-n7pwd" (UID: "49c51a2b-86c0-4f43-af87-0f9dc3a56986") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:25.648583 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:25.648315    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv podName:1f77f91a-1189-4c6c-b6cf-d5213e58ec43 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:41.648281195 +0000 UTC m=+34.173784333 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5sxv" (UniqueName: "kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv") pod "network-check-target-kpnng" (UID: "1f77f91a-1189-4c6c-b6cf-d5213e58ec43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:25.648583 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:25.648334    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs podName:e4069bb1-2913-45bf-ade6-79ab04c94fa8 nodeName:}" failed. No retries permitted until 2026-07-21 12:51:41.648322937 +0000 UTC m=+34.173826076 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs") pod "network-metrics-daemon-rjmvf" (UID: "e4069bb1-2913-45bf-ade6-79ab04c94fa8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:25.990680 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:25.990558    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:25.990680 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:25.990572    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:25.990680 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:25.990575    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:25.990913 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:25.990709    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:25.990913 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:25.990833    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:25.990913 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:25.990907    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:27.991394 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:27.991355    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:27.991838 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:27.991463    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:27.991838 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:27.991539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:27.991838 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:27.991640    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:27.991838 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:27.991676    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:27.991838 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:27.991740    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:29.990512 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:29.990137    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:29.990512 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:29.990421    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:29.990512 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:29.990435    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:29.990512 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:29.990459    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:29.990512 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:29.990506    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:29.991161 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:29.990581    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:30.095663 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.095630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5w7wb" event={"ID":"50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47","Type":"ContainerStarted","Data":"703fe667321742d587ac2ab0b512fdddbc1c99461a74dd36498205627f2bbf9c"}
Jul 21 12:51:30.097194 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.097167    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt" event={"ID":"ea43418c-d593-4ed0-b999-a1329a5caab3","Type":"ContainerStarted","Data":"3adc128a831b31efddd738afed52125f4ae60773a3813bb27960c99ccac1b1f0"}
Jul 21 12:51:30.098721 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.098697    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8f5wn" event={"ID":"b6b9e745-a2a2-4380-8f24-270804bc742f","Type":"ContainerStarted","Data":"61af194dc833197512263bae3adc14a01ee0552ac13f3e58776477dec2542309"}
Jul 21 12:51:30.100244 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.100220    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w92wm" event={"ID":"fcebc074-dea4-4fe1-9208-f01443f8720b","Type":"ContainerStarted","Data":"29b7ae506f303fb062cb3e1c5818a47586005ed1e095faf618c8ad2dd4e6c93a"}
Jul 21 12:51:30.101697 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.101675    2563 generic.go:356] "Generic (PLEG): container finished" podID="b1e589a5-672d-48e9-b4dc-aa3cbf2e551d" containerID="67a260c55d9555e3bc398ba59b1587e8afc20086c5711903db6f10df6977f0be" exitCode=0
Jul 21 12:51:30.101782 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.101742    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nlt9j" event={"ID":"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d","Type":"ContainerDied","Data":"67a260c55d9555e3bc398ba59b1587e8afc20086c5711903db6f10df6977f0be"}
Jul 21 12:51:30.103214 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.103169    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nln7n" event={"ID":"5bc0b65d-775f-43f5-b762-26ae86e132d1","Type":"ContainerStarted","Data":"397028ef42e482f48c3e14da6d5a1b66fcedea95e55fd33e9924e04ef1bf5644"}
Jul 21 12:51:30.104666 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.104643    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lhp7h" event={"ID":"69efe2b6-b508-459f-a900-baec7166c3ed","Type":"ContainerStarted","Data":"d31e3e034580830f8a94b32980b95a185728798de41caef27068c910b657f813"}
Jul 21 12:51:30.114223 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.114181    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5w7wb" podStartSLOduration=8.003494359 podStartE2EDuration="22.114167404s" totalImagesPullingTime="14.110673045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:30.113572525 +0000 UTC m=+22.639075727" watchObservedRunningTime="2026-07-21 12:51:30.114167404 +0000 UTC m=+22.639670559"
Jul 21 12:51:30.140626 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.140579    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lhp7h" podStartSLOduration=3.44115842 podStartE2EDuration="22.140565091s" totalImagesPullingTime="18.699406671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:30.140014843 +0000 UTC m=+22.665518001" watchObservedRunningTime="2026-07-21 12:51:30.140565091 +0000 UTC m=+22.666068245"
Jul 21 12:51:30.155469 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.155421    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nln7n" podStartSLOduration=3.488800764 podStartE2EDuration="22.155409832s" totalImagesPullingTime="18.666609068s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:30.155155572 +0000 UTC m=+22.680658725" watchObservedRunningTime="2026-07-21 12:51:30.155409832 +0000 UTC m=+22.680912985"
Jul 21 12:51:30.213227 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.213188    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8f5wn" podStartSLOduration=3.506620231 podStartE2EDuration="22.213173157s" totalImagesPullingTime="18.706552926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:30.212976698 +0000 UTC m=+22.738479855" watchObservedRunningTime="2026-07-21 12:51:30.213173157 +0000 UTC m=+22.738676311"
Jul 21 12:51:30.213501 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.213457    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-w92wm" podStartSLOduration=3.533993861 podStartE2EDuration="22.213448912s" totalImagesPullingTime="18.679455051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:30.193338735 +0000 UTC m=+22.718841913" watchObservedRunningTime="2026-07-21 12:51:30.213448912 +0000 UTC m=+22.738952066"
Jul 21 12:51:30.636523 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.636092    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5w7wb"
Jul 21 12:51:30.636890 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:30.636870    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5w7wb"
Jul 21 12:51:31.108111 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.108067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gcqc9" event={"ID":"2b2479d2-3197-427c-9dc0-da1b80bb6826","Type":"ContainerStarted","Data":"4a4567ed80572f8c7fde4711a7fcc4c6b977c138b331fd0d6481ed14b18ba86d"}
Jul 21 12:51:31.111717 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.111697    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 12:51:31.112086 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.112056    2563 generic.go:356] "Generic (PLEG): container finished" podID="cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b" containerID="32d5e81fba653ce996de0475179630c0878485a34de4f1d95ca8385ed8e18b26" exitCode=1
Jul 21 12:51:31.112195 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.112098    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9674" event={"ID":"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b","Type":"ContainerStarted","Data":"225bc41defa96887d271a248e387ac1e146d3e795397dc812ea4e10fa4ef882e"}
Jul 21 12:51:31.112195 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.112163    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9674" event={"ID":"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b","Type":"ContainerStarted","Data":"007ad7b2ff70ce5771c6d780803b212bdcf99e2ef255d8aea8cf2773eb7fa593"}
Jul 21 12:51:31.112310 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.112204    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9674" event={"ID":"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b","Type":"ContainerStarted","Data":"cffbc9ec17f8022d848221d8b20804762dea26470e4448e4b6f33b08965dae06"}
Jul 21 12:51:31.112310 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.112241    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9674" event={"ID":"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b","Type":"ContainerStarted","Data":"1186ce4dac4b07abe02ae19c526e24975dc73d81d139a86b3ffb95aa4920b33c"}
Jul 21 12:51:31.112310 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.112285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9674" event={"ID":"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b","Type":"ContainerStarted","Data":"c353483bceb1c37bb65f345688d1a76824fb78178919e74d1480e1d9513d5b5a"}
Jul 21 12:51:31.112457 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.112409    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9674" event={"ID":"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b","Type":"ContainerDied","Data":"32d5e81fba653ce996de0475179630c0878485a34de4f1d95ca8385ed8e18b26"}
Jul 21 12:51:31.134965 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.134860    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gcqc9" podStartSLOduration=4.438652344 podStartE2EDuration="23.134843094s" totalImagesPullingTime="18.69619075s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:31.134041673 +0000 UTC m=+23.659544829" watchObservedRunningTime="2026-07-21 12:51:31.134843094 +0000 UTC m=+23.660346248"
Jul 21 12:51:31.137580 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.137559    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5w7wb"
Jul 21 12:51:31.138109 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.138086    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5w7wb"
Jul 21 12:51:31.329650 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.329628    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 21 12:51:31.964736 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.964607    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T12:51:31.329647988Z","UUID":"cfc36368-e724-4a40-a35f-95e8622f8223","Handler":null,"Name":"","Endpoint":""}
Jul 21 12:51:31.966783 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.966417    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 21 12:51:31.966783 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.966463    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 21 12:51:31.990103 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.990058    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:31.990530 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.990064    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:31.990530 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:31.990199    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:31.990530 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:31.990071    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:31.990530 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:31.990267    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:31.990530 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:31.990330    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:32.116366 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:32.116327    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt" event={"ID":"ea43418c-d593-4ed0-b999-a1329a5caab3","Type":"ContainerStarted","Data":"92d5f517a7428191e55fc34ed43f2bcbdcac8124ab3b1609a7f0815b5e087dbc"}
Jul 21 12:51:33.122156 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:33.121994    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 12:51:33.122667 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:33.122596    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9674" event={"ID":"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b","Type":"ContainerStarted","Data":"68b2803a4a3c1ad97e5c9c9edc8c5606d26da292d444687dcbea324b1f27e8d4"}
Jul 21 12:51:33.124679 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:33.124652    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt" event={"ID":"ea43418c-d593-4ed0-b999-a1329a5caab3","Type":"ContainerStarted","Data":"c08f552cbe3dcab36fa3f0209e22504fda1a74141dd6e9cbc922fef1e9bc84a0"}
Jul 21 12:51:33.144540 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:33.144487    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dzgt" podStartSLOduration=3.896565706 podStartE2EDuration="25.144464776s" totalImagesPullingTime="21.24789907s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:51:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:33.14442954 +0000 UTC m=+25.669932695" watchObservedRunningTime="2026-07-21 12:51:33.144464776 +0000 UTC m=+25.669967930"
Jul 21 12:51:33.509758 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:33.509677    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:33.610993 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:33.610952    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:33.990693 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:33.990647    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:33.991272 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:33.990655    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:33.991893 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:33.991855    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:33.991968 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:33.991890    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:33.992122 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:33.992067    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:33.992216 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:33.992181    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:34.564031 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:34.564005    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:34.619005 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:34.618975    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:35.132230 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:35.132019    2563 generic.go:356] "Generic (PLEG): container finished" podID="b1e589a5-672d-48e9-b4dc-aa3cbf2e551d" containerID="fa5adcbe47cbf64f7da22b1f597b63bf5f4d54f37553f6e1b6072b097e3a0913" exitCode=0
Jul 21 12:51:35.132437 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:35.132070    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nlt9j" event={"ID":"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d","Type":"ContainerDied","Data":"fa5adcbe47cbf64f7da22b1f597b63bf5f4d54f37553f6e1b6072b097e3a0913"}
Jul 21 12:51:35.135853 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:35.135835    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 12:51:35.136223 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:35.136193    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9674" event={"ID":"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b","Type":"ContainerStarted","Data":"e104b9fa86bbaf6a2519f8a9703dd0c98b9ad9183b437b8770b2d33431b28fe8"}
Jul 21 12:51:35.136481 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:35.136460    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:35.136571 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:35.136522    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:35.136571 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:35.136568    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:35.136733 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:35.136712    2563 scope.go:117] "RemoveContainer" containerID="32d5e81fba653ce996de0475179630c0878485a34de4f1d95ca8385ed8e18b26"
Jul 21 12:51:35.154222 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:35.154202    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:35.155075 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:35.155057    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:51:35.990076 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:35.989997    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:35.990732 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:35.989997    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:35.990732 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:35.990130    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:35.990732 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:35.990006    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:35.990732 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:35.990225    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:35.990732 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:35.990359    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:36.142876 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:36.142849    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 12:51:36.143219 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:36.143196    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9674" event={"ID":"cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b","Type":"ContainerStarted","Data":"9ba9a2e4413952fb41d3eb796896c40fe368585e73abaeb6ce71321ed041822d"}
Jul 21 12:51:36.177487 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:36.177437    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-g9674" podStartSLOduration=9.126198168 podStartE2EDuration="28.177422568s" totalImagesPullingTime="19.0512244s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:36.176929426 +0000 UTC m=+28.702432581" watchObservedRunningTime="2026-07-21 12:51:36.177422568 +0000 UTC m=+28.702925723"
Jul 21 12:51:36.793812 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:36.793736    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kpnng"]
Jul 21 12:51:36.794002 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:36.793854    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:36.794002 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:36.793963    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:36.794606 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:36.794571    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rjmvf"]
Jul 21 12:51:36.794708 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:36.794661    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:36.794785 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:36.794766    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:36.817499 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:36.817468    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n7pwd"]
Jul 21 12:51:36.817621 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:36.817556    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:36.817673 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:36.817644    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:37.148248 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:37.148086    2563 generic.go:356] "Generic (PLEG): container finished" podID="b1e589a5-672d-48e9-b4dc-aa3cbf2e551d" containerID="f6365a5a448bd0e08c8f495ffc7e8bf0b74ec95309b2c549753b09ac22e9eea3" exitCode=0
Jul 21 12:51:37.148750 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:37.148161    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nlt9j" event={"ID":"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d","Type":"ContainerDied","Data":"f6365a5a448bd0e08c8f495ffc7e8bf0b74ec95309b2c549753b09ac22e9eea3"}
Jul 21 12:51:38.989745 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:38.989690    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:38.989745 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:38.989745    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:38.990378 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:38.989745    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:38.990378 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:38.989807    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:38.990378 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:38.989947    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:38.990378 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:38.990032    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:39.157885 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:39.157795    2563 generic.go:356] "Generic (PLEG): container finished" podID="b1e589a5-672d-48e9-b4dc-aa3cbf2e551d" containerID="52750915a9ad8e89d9737cbf9e543696826edfebc6356f3f4353c9da890ebe7a" exitCode=0
Jul 21 12:51:39.157885 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:39.157873    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nlt9j" event={"ID":"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d","Type":"ContainerDied","Data":"52750915a9ad8e89d9737cbf9e543696826edfebc6356f3f4353c9da890ebe7a"}
Jul 21 12:51:40.990398 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:40.990319    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:40.990398 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:40.990359    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:40.990398 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:40.990387    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:40.991043 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:40.990494    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7pwd" podUID="49c51a2b-86c0-4f43-af87-0f9dc3a56986"
Jul 21 12:51:40.991043 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:40.990624    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kpnng" podUID="1f77f91a-1189-4c6c-b6cf-d5213e58ec43"
Jul 21 12:51:40.991043 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:40.990733    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjmvf" podUID="e4069bb1-2913-45bf-ade6-79ab04c94fa8"
Jul 21 12:51:41.669383 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:41.669348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:41.669569 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:41.669395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs\") pod \"network-metrics-daemon-rjmvf\" (UID: \"e4069bb1-2913-45bf-ade6-79ab04c94fa8\") " pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:41.669569 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:41.669427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5sxv\" (UniqueName: \"kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv\") pod \"network-check-target-kpnng\" (UID: \"1f77f91a-1189-4c6c-b6cf-d5213e58ec43\") " pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:41.669569 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:41.669504    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:41.669569 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:41.669569    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret podName:49c51a2b-86c0-4f43-af87-0f9dc3a56986 nodeName:}" failed. No retries permitted until 2026-07-21 12:52:13.669550049 +0000 UTC m=+66.195053202 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret") pod "global-pull-secret-syncer-n7pwd" (UID: "49c51a2b-86c0-4f43-af87-0f9dc3a56986") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:51:41.669852 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:41.669566    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:51:41.669852 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:41.669589    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:51:41.669852 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:41.669597    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:41.669852 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:41.669663    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs podName:e4069bb1-2913-45bf-ade6-79ab04c94fa8 nodeName:}" failed. No retries permitted until 2026-07-21 12:52:13.669643611 +0000 UTC m=+66.195146744 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs") pod "network-metrics-daemon-rjmvf" (UID: "e4069bb1-2913-45bf-ade6-79ab04c94fa8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:51:41.669852 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:41.669601    2563 projected.go:196] Error preparing data for projected volume kube-api-access-m5sxv for pod openshift-network-diagnostics/network-check-target-kpnng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:41.669852 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:41.669718    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv podName:1f77f91a-1189-4c6c-b6cf-d5213e58ec43 nodeName:}" failed. No retries permitted until 2026-07-21 12:52:13.66970612 +0000 UTC m=+66.195209254 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5sxv" (UniqueName: "kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv") pod "network-check-target-kpnng" (UID: "1f77f91a-1189-4c6c-b6cf-d5213e58ec43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:51:41.760757 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:41.760714    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:41.856880 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:41.856849    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:42.418797 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.418728    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:42.499170 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.499080    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:42.559874 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.559834    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:42.667683 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.667646    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:42.726984 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.726946    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:42.784804 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.784727    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-74.ec2.internal" event="NodeReady"
Jul 21 12:51:42.784931 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.784870    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 21 12:51:42.827481 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.827441    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rtn7b"]
Jul 21 12:51:42.832774 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.832742    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b6h6z"]
Jul 21 12:51:42.832929 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.832872    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:42.835264 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.835243    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 21 12:51:42.835378 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.835276    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 21 12:51:42.835546 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.835531    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kcsl4\""
Jul 21 12:51:42.836369 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.836347    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b6h6z"
Jul 21 12:51:42.838663 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.838621    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 21 12:51:42.839018 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.839000    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 21 12:51:42.839199 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.839184    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 21 12:51:42.839512 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.839373    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kg652\""
Jul 21 12:51:42.840448 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.840393    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rtn7b"]
Jul 21 12:51:42.842717 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.842559    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b6h6z"]
Jul 21 12:51:42.892171 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.892083    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:42.980530 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.980496    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnskj\" (UniqueName: \"kubernetes.io/projected/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-kube-api-access-qnskj\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:42.980702 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.980547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9z5c\" (UniqueName: \"kubernetes.io/projected/2726d88e-bdde-4182-a2a5-2a1224965db8-kube-api-access-b9z5c\") pod \"ingress-canary-b6h6z\" (UID: \"2726d88e-bdde-4182-a2a5-2a1224965db8\") " pod="openshift-ingress-canary/ingress-canary-b6h6z"
Jul 21 12:51:42.980702 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.980575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2726d88e-bdde-4182-a2a5-2a1224965db8-cert\") pod \"ingress-canary-b6h6z\" (UID: \"2726d88e-bdde-4182-a2a5-2a1224965db8\") " pod="openshift-ingress-canary/ingress-canary-b6h6z"
Jul 21 12:51:42.980702 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.980611    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-config-volume\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:42.980702 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.980644    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:42.980902 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.980729    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-tmp-dir\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:42.986935 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.986912    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:42.990763 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.990742    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:51:42.990860 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.990850    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:51:42.990960 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.990942    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:51:42.993730 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.993700    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 12:51:42.993730 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.993716    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rtsf5\""
Jul 21 12:51:42.993879 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.993756    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5ccp9\""
Jul 21 12:51:42.993879 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.993803    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 12:51:42.993879 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.993703    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 12:51:42.994050 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:42.994032    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 12:51:43.081529 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.081434    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnskj\" (UniqueName: \"kubernetes.io/projected/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-kube-api-access-qnskj\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:43.081529 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.081495    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9z5c\" (UniqueName: \"kubernetes.io/projected/2726d88e-bdde-4182-a2a5-2a1224965db8-kube-api-access-b9z5c\") pod \"ingress-canary-b6h6z\" (UID: \"2726d88e-bdde-4182-a2a5-2a1224965db8\") " pod="openshift-ingress-canary/ingress-canary-b6h6z"
Jul 21 12:51:43.081529 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.081526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2726d88e-bdde-4182-a2a5-2a1224965db8-cert\") pod \"ingress-canary-b6h6z\" (UID: \"2726d88e-bdde-4182-a2a5-2a1224965db8\") " pod="openshift-ingress-canary/ingress-canary-b6h6z"
Jul 21 12:51:43.081793 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.081562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-config-volume\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:43.081793 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.081583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:43.081793 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.081626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-tmp-dir\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:43.081937 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:43.081805    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:51:43.081937 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:43.081878    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls podName:698e2686-c7ef-4e97-bed4-7cc24e1bd4fe nodeName:}" failed. No retries permitted until 2026-07-21 12:51:43.581854508 +0000 UTC m=+36.107357655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls") pod "dns-default-rtn7b" (UID: "698e2686-c7ef-4e97-bed4-7cc24e1bd4fe") : secret "dns-default-metrics-tls" not found
Jul 21 12:51:43.082044 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.081967    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-tmp-dir\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:43.082413 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.082388    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-config-volume\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:43.086413 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.086392    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2726d88e-bdde-4182-a2a5-2a1224965db8-cert\") pod \"ingress-canary-b6h6z\" (UID: \"2726d88e-bdde-4182-a2a5-2a1224965db8\") " pod="openshift-ingress-canary/ingress-canary-b6h6z"
Jul 21 12:51:43.090135 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.090087    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnskj\" (UniqueName: \"kubernetes.io/projected/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-kube-api-access-qnskj\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:43.091171 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.091151    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9z5c\" (UniqueName: \"kubernetes.io/projected/2726d88e-bdde-4182-a2a5-2a1224965db8-kube-api-access-b9z5c\") pod \"ingress-canary-b6h6z\" (UID: \"2726d88e-bdde-4182-a2a5-2a1224965db8\") " pod="openshift-ingress-canary/ingress-canary-b6h6z"
Jul 21 12:51:43.153108 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.153071    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b6h6z"
Jul 21 12:51:43.386811 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.386621    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:43.486929 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.486893    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:43.585351 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.585315    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:43.585526 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:43.585496    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:51:43.585600 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:43.585581    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls podName:698e2686-c7ef-4e97-bed4-7cc24e1bd4fe nodeName:}" failed. No retries permitted until 2026-07-21 12:51:44.585564592 +0000 UTC m=+37.111067725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls") pod "dns-default-rtn7b" (UID: "698e2686-c7ef-4e97-bed4-7cc24e1bd4fe") : secret "dns-default-metrics-tls" not found
Jul 21 12:51:43.743122 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.743086    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:43.803011 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.802974    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b6h6z"]
Jul 21 12:51:43.806217 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:43.806188    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2726d88e_bdde_4182_a2a5_2a1224965db8.slice/crio-aeaf98fc63cae99e1fb9c08e10f7788d4ec61e7947066e6d9379cc830eef8e36 WatchSource:0}: Error finding container aeaf98fc63cae99e1fb9c08e10f7788d4ec61e7947066e6d9379cc830eef8e36: Status 404 returned error can't find the container with id aeaf98fc63cae99e1fb9c08e10f7788d4ec61e7947066e6d9379cc830eef8e36
Jul 21 12:51:43.808957 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:43.808821    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:44.172764 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.172680    2563 generic.go:356] "Generic (PLEG): container finished" podID="b1e589a5-672d-48e9-b4dc-aa3cbf2e551d" containerID="7c92651dd8269906bf92df2be66adf4d82e1145e1e0e7f932b5719e47c5673cc" exitCode=0
Jul 21 12:51:44.172935 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.172759    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nlt9j" event={"ID":"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d","Type":"ContainerDied","Data":"7c92651dd8269906bf92df2be66adf4d82e1145e1e0e7f932b5719e47c5673cc"}
Jul 21 12:51:44.173796 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.173771    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b6h6z" event={"ID":"2726d88e-bdde-4182-a2a5-2a1224965db8","Type":"ContainerStarted","Data":"aeaf98fc63cae99e1fb9c08e10f7788d4ec61e7947066e6d9379cc830eef8e36"}
Jul 21 12:51:44.175546 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.175512    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:44.495904 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.495832    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"]
Jul 21 12:51:44.498752 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.498729    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"
Jul 21 12:51:44.501423 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.501395    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 21 12:51:44.501526 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.501400    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 21 12:51:44.501526 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.501498    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5plqh\""
Jul 21 12:51:44.508678 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.508658    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"]
Jul 21 12:51:44.594473 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.594437    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-qhqc5\" (UID: \"f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"
Jul 21 12:51:44.594666 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.594597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:44.594666 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.594644    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-qhqc5\" (UID: \"f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"
Jul 21 12:51:44.594775 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:44.594711    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:51:44.594822 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:44.594788    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls podName:698e2686-c7ef-4e97-bed4-7cc24e1bd4fe nodeName:}" failed. No retries permitted until 2026-07-21 12:51:46.59476862 +0000 UTC m=+39.120271764 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls") pod "dns-default-rtn7b" (UID: "698e2686-c7ef-4e97-bed4-7cc24e1bd4fe") : secret "dns-default-metrics-tls" not found
Jul 21 12:51:44.695644 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.695600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-qhqc5\" (UID: \"f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"
Jul 21 12:51:44.695812 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.695735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-qhqc5\" (UID: \"f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"
Jul 21 12:51:44.695812 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:44.695795    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 12:51:44.695928 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:44.695878    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert podName:f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee nodeName:}" failed. No retries permitted until 2026-07-21 12:51:45.195857077 +0000 UTC m=+37.721360227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-qhqc5" (UID: "f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee") : secret "networking-console-plugin-cert" not found
Jul 21 12:51:44.696471 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.696449    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-qhqc5\" (UID: \"f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"
Jul 21 12:51:44.802142 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.802056    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:44.961721 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.961689    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-bj45r"]
Jul 21 12:51:44.964639 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.964611    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-bj45r"
Jul 21 12:51:44.967547 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.967524    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 21 12:51:44.967857 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.967837    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 21 12:51:44.968044 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.968024    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-54fxv\""
Jul 21 12:51:44.968865 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.968845    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 21 12:51:44.968953 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.968903    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 21 12:51:44.972045 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:44.972023    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-bj45r"]
Jul 21 12:51:45.088130 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.088038    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:45.099036 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.099007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e6cb5a57-6165-4e1d-9dda-c08ce52ded1f-signing-key\") pod \"service-ca-58598695f7-bj45r\" (UID: \"e6cb5a57-6165-4e1d-9dda-c08ce52ded1f\") " pod="openshift-service-ca/service-ca-58598695f7-bj45r"
Jul 21 12:51:45.099195 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.099046    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6c8b\" (UniqueName: \"kubernetes.io/projected/e6cb5a57-6165-4e1d-9dda-c08ce52ded1f-kube-api-access-l6c8b\") pod \"service-ca-58598695f7-bj45r\" (UID: \"e6cb5a57-6165-4e1d-9dda-c08ce52ded1f\") " pod="openshift-service-ca/service-ca-58598695f7-bj45r"
Jul 21 12:51:45.099253 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.099182    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e6cb5a57-6165-4e1d-9dda-c08ce52ded1f-signing-cabundle\") pod \"service-ca-58598695f7-bj45r\" (UID: \"e6cb5a57-6165-4e1d-9dda-c08ce52ded1f\") " pod="openshift-service-ca/service-ca-58598695f7-bj45r"
Jul 21 12:51:45.180214 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.180166    2563 generic.go:356] "Generic (PLEG): container finished" podID="b1e589a5-672d-48e9-b4dc-aa3cbf2e551d" containerID="7071f9f2bd6a702a641bfee4fa92d618ff0192c317e54660ea20cef36448e01c" exitCode=0
Jul 21 12:51:45.180392 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.180217    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nlt9j" event={"ID":"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d","Type":"ContainerDied","Data":"7071f9f2bd6a702a641bfee4fa92d618ff0192c317e54660ea20cef36448e01c"}
Jul 21 12:51:45.188737 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.188705    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:45.199655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.199631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e6cb5a57-6165-4e1d-9dda-c08ce52ded1f-signing-key\") pod \"service-ca-58598695f7-bj45r\" (UID: \"e6cb5a57-6165-4e1d-9dda-c08ce52ded1f\") " pod="openshift-service-ca/service-ca-58598695f7-bj45r"
Jul 21 12:51:45.199786 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.199666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6c8b\" (UniqueName: \"kubernetes.io/projected/e6cb5a57-6165-4e1d-9dda-c08ce52ded1f-kube-api-access-l6c8b\") pod \"service-ca-58598695f7-bj45r\" (UID: \"e6cb5a57-6165-4e1d-9dda-c08ce52ded1f\") " pod="openshift-service-ca/service-ca-58598695f7-bj45r"
Jul 21 12:51:45.199786 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.199695    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-qhqc5\" (UID: \"f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"
Jul 21 12:51:45.199878 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:45.199789    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 12:51:45.199878 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:45.199853    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert podName:f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee nodeName:}" failed. No retries permitted until 2026-07-21 12:51:46.199835841 +0000 UTC m=+38.725338994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-qhqc5" (UID: "f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee") : secret "networking-console-plugin-cert" not found
Jul 21 12:51:45.199959 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.199888    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e6cb5a57-6165-4e1d-9dda-c08ce52ded1f-signing-cabundle\") pod \"service-ca-58598695f7-bj45r\" (UID: \"e6cb5a57-6165-4e1d-9dda-c08ce52ded1f\") " pod="openshift-service-ca/service-ca-58598695f7-bj45r"
Jul 21 12:51:45.200533 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.200516    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e6cb5a57-6165-4e1d-9dda-c08ce52ded1f-signing-cabundle\") pod \"service-ca-58598695f7-bj45r\" (UID: \"e6cb5a57-6165-4e1d-9dda-c08ce52ded1f\") " pod="openshift-service-ca/service-ca-58598695f7-bj45r"
Jul 21 12:51:45.204440 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.204419    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e6cb5a57-6165-4e1d-9dda-c08ce52ded1f-signing-key\") pod \"service-ca-58598695f7-bj45r\" (UID: \"e6cb5a57-6165-4e1d-9dda-c08ce52ded1f\") " pod="openshift-service-ca/service-ca-58598695f7-bj45r"
Jul 21 12:51:45.211879 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.211854    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6c8b\" (UniqueName: \"kubernetes.io/projected/e6cb5a57-6165-4e1d-9dda-c08ce52ded1f-kube-api-access-l6c8b\") pod \"service-ca-58598695f7-bj45r\" (UID: \"e6cb5a57-6165-4e1d-9dda-c08ce52ded1f\") " pod="openshift-service-ca/service-ca-58598695f7-bj45r"
Jul 21 12:51:45.275956 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.275903    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-bj45r"
Jul 21 12:51:45.289409 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.289379    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:45.980134 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.980105    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-bj45r"]
Jul 21 12:51:45.982563 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:45.982533    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6cb5a57_6165_4e1d_9dda_c08ce52ded1f.slice/crio-b8cb52d693c973ce76e2111eb363c0209d40389ac1b38709caccecea874bee2a WatchSource:0}: Error finding container b8cb52d693c973ce76e2111eb363c0209d40389ac1b38709caccecea874bee2a: Status 404 returned error can't find the container with id b8cb52d693c973ce76e2111eb363c0209d40389ac1b38709caccecea874bee2a
Jul 21 12:51:45.996830 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:45.996793    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:46.184017 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:46.183984    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b6h6z" event={"ID":"2726d88e-bdde-4182-a2a5-2a1224965db8","Type":"ContainerStarted","Data":"c928b21e359ffe84b487dbe06ad005f8afd4444f598e6102859734d534de57cd"}
Jul 21 12:51:46.185085 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:46.185058    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-bj45r" event={"ID":"e6cb5a57-6165-4e1d-9dda-c08ce52ded1f","Type":"ContainerStarted","Data":"b8cb52d693c973ce76e2111eb363c0209d40389ac1b38709caccecea874bee2a"}
Jul 21 12:51:46.188426 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:46.188403    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nlt9j" event={"ID":"b1e589a5-672d-48e9-b4dc-aa3cbf2e551d","Type":"ContainerStarted","Data":"44dc659c665948eccce360921cae7954b975462b1e7ecf85b6395899fb3a4059"}
Jul 21 12:51:46.206052 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:46.205999    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b6h6z" podStartSLOduration=1.950229779 podStartE2EDuration="4.205981002s" totalImagesPullingTime="2.255751223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:46.205356018 +0000 UTC m=+38.730859191" watchObservedRunningTime="2026-07-21 12:51:46.205981002 +0000 UTC m=+38.731484157"
Jul 21 12:51:46.208697 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:46.208665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-qhqc5\" (UID: \"f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"
Jul 21 12:51:46.208841 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:46.208822    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 12:51:46.208924 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:46.208913    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert podName:f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee nodeName:}" failed. No retries permitted until 2026-07-21 12:51:48.208892772 +0000 UTC m=+40.734395931 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-qhqc5" (UID: "f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee") : secret "networking-console-plugin-cert" not found
Jul 21 12:51:46.235482 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:46.235424    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nlt9j" podStartSLOduration=8.050264403 podStartE2EDuration="38.23540297s" totalImagesPullingTime="30.185138567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:51:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:46.233821362 +0000 UTC m=+38.759324516" watchObservedRunningTime="2026-07-21 12:51:46.23540297 +0000 UTC m=+38.760906124"
Jul 21 12:51:46.236392 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:46.236364    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:46.307265 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:46.307229    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:46.611511 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:46.611340    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:46.611511 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:46.611499    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:51:46.611710 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:46.611564    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls podName:698e2686-c7ef-4e97-bed4-7cc24e1bd4fe nodeName:}" failed. No retries permitted until 2026-07-21 12:51:50.611548085 +0000 UTC m=+43.137051218 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls") pod "dns-default-rtn7b" (UID: "698e2686-c7ef-4e97-bed4-7cc24e1bd4fe") : secret "dns-default-metrics-tls" not found
Jul 21 12:51:48.223608 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:48.223560    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-qhqc5\" (UID: \"f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"
Jul 21 12:51:48.224050 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:48.223738    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 12:51:48.224050 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:48.223827    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert podName:f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee nodeName:}" failed. No retries permitted until 2026-07-21 12:51:52.22380397 +0000 UTC m=+44.749307121 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-qhqc5" (UID: "f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee") : secret "networking-console-plugin-cert" not found
Jul 21 12:51:48.318870 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:48.318828    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:48.419724 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:48.419671    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:49.197287 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:49.197250    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-bj45r" event={"ID":"e6cb5a57-6165-4e1d-9dda-c08ce52ded1f","Type":"ContainerStarted","Data":"a9ed1ace56a2d0f693fa0cd44d149648810f6f67f9be6784c6a4665eb50c7f7d"}
Jul 21 12:51:49.216616 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:49.216567    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-bj45r" podStartSLOduration=2.67678307 podStartE2EDuration="5.216550776s" totalImagesPullingTime="2.539767706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:49.215741952 +0000 UTC m=+41.741245107" watchObservedRunningTime="2026-07-21 12:51:49.216550776 +0000 UTC m=+41.742053931"
Jul 21 12:51:49.239129 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:49.239102    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:50.133208 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:50.133175    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:50.234413 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:50.234371    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:50.552338 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:50.552213    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:50.611480 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:50.611439    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:50.642728 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:50.642686    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:50.642936 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:50.642856    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:51:50.642993 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:50.642948    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls podName:698e2686-c7ef-4e97-bed4-7cc24e1bd4fe nodeName:}" failed. No retries permitted until 2026-07-21 12:51:58.642926376 +0000 UTC m=+51.168429513 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls") pod "dns-default-rtn7b" (UID: "698e2686-c7ef-4e97-bed4-7cc24e1bd4fe") : secret "dns-default-metrics-tls" not found
Jul 21 12:51:50.818502 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:50.818409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:51.616233 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:51.616202    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:51.831513 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:51.831482    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:52.217713 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:52.217684    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:52.256685 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:52.256651    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-qhqc5\" (UID: \"f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"
Jul 21 12:51:52.256841 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:52.256798    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 21 12:51:52.256881 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:51:52.256863    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert podName:f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee nodeName:}" failed. No retries permitted until 2026-07-21 12:52:00.256847643 +0000 UTC m=+52.782350780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-qhqc5" (UID: "f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee") : secret "networking-console-plugin-cert" not found
Jul 21 12:51:52.420788 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:52.420753    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:52.625371 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:52.625341    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:52.826605 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:52.826572    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:53.812908 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:53.812872    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:53.871875 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:53.871834    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:54.138525 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:54.138487    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:54.239906 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:54.239851    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:54.418370 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:54.418268    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:54.626809 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:54.626777    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:55.032619 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:55.032586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:55.127084 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:55.127024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:55.616598 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:55.616562    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:56.021363 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:56.021256    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:56.461175 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:56.461145    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:56.829887 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:56.829802    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:57.020241 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:57.020205    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:57.427129 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:57.427095    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:57.490304 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:57.490251    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:57.666023 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:57.665991    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:58.011132 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:58.011096    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:58.621328 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:58.621275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:58.708080 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:58.708043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:58.711569 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:58.711544    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/698e2686-c7ef-4e97-bed4-7cc24e1bd4fe-metrics-tls\") pod \"dns-default-rtn7b\" (UID: \"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe\") " pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:58.748100 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:58.748066    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:51:58.816873 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:58.816841    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:58.911346 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:58.911244    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rtn7b"]
Jul 21 12:51:58.914725 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:51:58.914698    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod698e2686_c7ef_4e97_bed4_7cc24e1bd4fe.slice/crio-96a2e3b90ee030f2b95e13df6fa8e6341cae70d0bd97f4c76782fdcbaccd3fe6 WatchSource:0}: Error finding container 96a2e3b90ee030f2b95e13df6fa8e6341cae70d0bd97f4c76782fdcbaccd3fe6: Status 404 returned error can't find the container with id 96a2e3b90ee030f2b95e13df6fa8e6341cae70d0bd97f4c76782fdcbaccd3fe6
Jul 21 12:51:59.220027 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:59.219947    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:59.226447 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:59.226416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rtn7b" event={"ID":"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe","Type":"ContainerStarted","Data":"96a2e3b90ee030f2b95e13df6fa8e6341cae70d0bd97f4c76782fdcbaccd3fe6"}
Jul 21 12:51:59.413108 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:59.413067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:51:59.617869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:51:59.617837    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:00.217957 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:00.217914    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:00.323041 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:00.323004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-qhqc5\" (UID: \"f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"
Jul 21 12:52:00.325587 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:00.325559    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-qhqc5\" (UID: \"f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"
Jul 21 12:52:00.408993 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:00.408866    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"
Jul 21 12:52:00.438030 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:00.438000    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:00.552996 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:00.552966    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5"]
Jul 21 12:52:00.556883 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:52:00.556837    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b7850b_0f49_4bb4_8de3_4a8a2396a9ee.slice/crio-0cc75067208e56186970103fbece553e77c8c881ffe36f96ddcd628a61ae61fd WatchSource:0}: Error finding container 0cc75067208e56186970103fbece553e77c8c881ffe36f96ddcd628a61ae61fd: Status 404 returned error can't find the container with id 0cc75067208e56186970103fbece553e77c8c881ffe36f96ddcd628a61ae61fd
Jul 21 12:52:01.113825 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:01.113780    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:01.215087 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:01.215058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:01.232224 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:01.232191    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5" event={"ID":"f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee","Type":"ContainerStarted","Data":"0cc75067208e56186970103fbece553e77c8c881ffe36f96ddcd628a61ae61fd"}
Jul 21 12:52:01.283876 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:01.283836    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:01.402263 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:01.402167    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:01.794687 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:01.794644    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:01.862259 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:01.862212    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:02.011339 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:02.011221    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:02.082778 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:02.082748    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:02.240741 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:02.240697    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rtn7b" event={"ID":"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe","Type":"ContainerStarted","Data":"1b40329cacf52d54c81ea5fe03f94e0511527fbbe5381b108a1afc35cfcceb09"}
Jul 21 12:52:02.241164 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:02.240788    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rtn7b" event={"ID":"698e2686-c7ef-4e97-bed4-7cc24e1bd4fe","Type":"ContainerStarted","Data":"c231a30b2a464a959b6e3dd36d9616cca962fe28b3a4f068640b628b630cfcb3"}
Jul 21 12:52:02.241164 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:02.240819    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:52:02.242208 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:02.242184    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5" event={"ID":"f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee","Type":"ContainerStarted","Data":"1ba641358221d568d4a6a287424aedfe9612ddec47745bc9ea3d9690695cd7aa"}
Jul 21 12:52:02.258917 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:02.258864    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rtn7b" podStartSLOduration=17.679149765 podStartE2EDuration="20.2588489s" totalImagesPullingTime="2.579699135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:02.258072553 +0000 UTC m=+54.783575730" watchObservedRunningTime="2026-07-21 12:52:02.2588489 +0000 UTC m=+54.784352102"
Jul 21 12:52:02.276033 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:02.275929    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qhqc5" podStartSLOduration=16.708732117 podStartE2EDuration="18.275911691s" totalImagesPullingTime="1.567179574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:02.274525993 +0000 UTC m=+54.800029150" watchObservedRunningTime="2026-07-21 12:52:02.275911691 +0000 UTC m=+54.801414824"
Jul 21 12:52:02.990287 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:02.990239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:03.190086 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:03.190040    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:03.324108 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:03.324018    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:04.008169 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:04.008138    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:04.109649 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:04.109616    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:04.338178 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:04.338089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:04.390552 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:04.390523    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:06.947047 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.947000    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:06.947445 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.947120    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"]
Jul 21 12:52:06.976465 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.976427    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w"]
Jul 21 12:52:06.976603 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.976558    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:06.980849 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.980827    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 21 12:52:06.980849 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.980827    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 21 12:52:06.981088 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.980901    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jjwbz\""
Jul 21 12:52:06.981088 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.981028    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 21 12:52:06.989195 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.989175    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 21 12:52:06.990579 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.990563    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"]
Jul 21 12:52:06.990684 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.990669    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w"
Jul 21 12:52:06.994354 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.994334    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 21 12:52:06.994478 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.994460    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 21 12:52:06.994628 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.994613    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-tpsnb\""
Jul 21 12:52:06.994941 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.994927    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 21 12:52:06.995394 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:06.995375    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 21 12:52:07.012813 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.012789    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w"]
Jul 21 12:52:07.012813 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.012817    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"]
Jul 21 12:52:07.013042 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.012836    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"]
Jul 21 12:52:07.013042 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.012922    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"
Jul 21 12:52:07.016064 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.016034    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 21 12:52:07.032015 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.031987    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"]
Jul 21 12:52:07.047578 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.047547    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-d5ptb"]
Jul 21 12:52:07.047733 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.047667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"
Jul 21 12:52:07.051484 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.051461    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 21 12:52:07.053185 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.053159    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 21 12:52:07.053358 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.053166    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 21 12:52:07.053449 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.053167    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-8756m\""
Jul 21 12:52:07.053510 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.053215    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 21 12:52:07.060209 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.060186    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-pnxvv"]
Jul 21 12:52:07.060353 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.060334    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-d5ptb"
Jul 21 12:52:07.063383 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.063358    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 21 12:52:07.063513 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.063395    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-p2p2p\""
Jul 21 12:52:07.063513 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.063359    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 21 12:52:07.070200 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-trusted-ca\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.070340 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-image-registry-private-configuration\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.070340 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-registry-tls\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.070340 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070245    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjd57\" (UniqueName: \"kubernetes.io/projected/9d55962c-a472-465a-8285-9c8bf0856ae4-kube-api-access-pjd57\") pod \"downloads-5bc67dcd46-d5ptb\" (UID: \"9d55962c-a472-465a-8285-9c8bf0856ae4\") " pod="openshift-console/downloads-5bc67dcd46-d5ptb"
Jul 21 12:52:07.070340 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070280    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-installation-pull-secrets\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.070340 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e7078810-2861-4a0f-a12c-16e80f3f2319-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cd9b6688d-zrw92\" (UID: \"e7078810-2861-4a0f-a12c-16e80f3f2319\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"
Jul 21 12:52:07.070571 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crzcz\" (UniqueName: \"kubernetes.io/projected/e7078810-2861-4a0f-a12c-16e80f3f2319-kube-api-access-crzcz\") pod \"klusterlet-addon-workmgr-5cd9b6688d-zrw92\" (UID: \"e7078810-2861-4a0f-a12c-16e80f3f2319\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"
Jul 21 12:52:07.070571 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55sdr\" (UniqueName: \"kubernetes.io/projected/41f99d1e-551f-457e-8a74-87f9702496cf-kube-api-access-55sdr\") pod \"cluster-monitoring-operator-76cf8966f8-b5v7d\" (UID: \"41f99d1e-551f-457e-8a74-87f9702496cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"
Jul 21 12:52:07.070571 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070511    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-ca-trust-extracted\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.070571 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070529    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vjvn\" (UniqueName: \"kubernetes.io/projected/3303870c-5cda-4ec0-b6df-6b54455ee748-kube-api-access-2vjvn\") pod \"managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w\" (UID: \"3303870c-5cda-4ec0-b6df-6b54455ee748\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w"
Jul 21 12:52:07.070571 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67ldz\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-kube-api-access-67ldz\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.070571 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7078810-2861-4a0f-a12c-16e80f3f2319-tmp\") pod \"klusterlet-addon-workmgr-5cd9b6688d-zrw92\" (UID: \"e7078810-2861-4a0f-a12c-16e80f3f2319\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"
Jul 21 12:52:07.070791 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/41f99d1e-551f-457e-8a74-87f9702496cf-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-b5v7d\" (UID: \"41f99d1e-551f-457e-8a74-87f9702496cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"
Jul 21 12:52:07.070791 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070657    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-bound-sa-token\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.070791 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070681    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41f99d1e-551f-457e-8a74-87f9702496cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-b5v7d\" (UID: \"41f99d1e-551f-457e-8a74-87f9702496cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"
Jul 21 12:52:07.070791 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070700    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3303870c-5cda-4ec0-b6df-6b54455ee748-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w\" (UID: \"3303870c-5cda-4ec0-b6df-6b54455ee748\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w"
Jul 21 12:52:07.070791 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.070720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-registry-certificates\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.081255 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.081228    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:07.086344 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.086321    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-785f46d866-4qbft"]
Jul 21 12:52:07.086467 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.086445    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.090574 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.090551    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 21 12:52:07.090574 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.090564    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 21 12:52:07.090737 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.090598    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 21 12:52:07.091229 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.091210    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 21 12:52:07.091341 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.091284    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5n268\""
Jul 21 12:52:07.095773 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.095754    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 21 12:52:07.103706 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.103676    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"]
Jul 21 12:52:07.103864 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.103716    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"]
Jul 21 12:52:07.103864 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.103733    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-d5ptb"]
Jul 21 12:52:07.103864 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.103747    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-pnxvv"]
Jul 21 12:52:07.103864 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.103760    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-785f46d866-4qbft"]
Jul 21 12:52:07.103864 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.103806    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.104105 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:07.103913    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-67ldz registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf" podUID="ea2c78bd-ca63-45d3-bd9a-b610c7430ae2"
Jul 21 12:52:07.106644 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.106624    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 21 12:52:07.106823 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.106805    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kwf9p\""
Jul 21 12:52:07.106823 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.106819    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 21 12:52:07.106940 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.106832    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 21 12:52:07.107100 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.107082    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 21 12:52:07.107153 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.107086    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 21 12:52:07.107211 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.107197    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 21 12:52:07.165972 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.165944    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g9674"
Jul 21 12:52:07.171264 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171241    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-default-certificate\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.171376 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-image-registry-private-configuration\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.171376 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171315    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-registry-tls\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.171376 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171346    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjd57\" (UniqueName: \"kubernetes.io/projected/9d55962c-a472-465a-8285-9c8bf0856ae4-kube-api-access-pjd57\") pod \"downloads-5bc67dcd46-d5ptb\" (UID: \"9d55962c-a472-465a-8285-9c8bf0856ae4\") " pod="openshift-console/downloads-5bc67dcd46-d5ptb"
Jul 21 12:52:07.171376 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171362    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-installation-pull-secrets\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.171560 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e7078810-2861-4a0f-a12c-16e80f3f2319-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cd9b6688d-zrw92\" (UID: \"e7078810-2861-4a0f-a12c-16e80f3f2319\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"
Jul 21 12:52:07.171560 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crzcz\" (UniqueName: \"kubernetes.io/projected/e7078810-2861-4a0f-a12c-16e80f3f2319-kube-api-access-crzcz\") pod \"klusterlet-addon-workmgr-5cd9b6688d-zrw92\" (UID: \"e7078810-2861-4a0f-a12c-16e80f3f2319\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"
Jul 21 12:52:07.171657 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171586    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-metrics-certs\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.171770 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55sdr\" (UniqueName: \"kubernetes.io/projected/41f99d1e-551f-457e-8a74-87f9702496cf-kube-api-access-55sdr\") pod \"cluster-monitoring-operator-76cf8966f8-b5v7d\" (UID: \"41f99d1e-551f-457e-8a74-87f9702496cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"
Jul 21 12:52:07.171888 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171871    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d12b1d08-a810-4d9a-bcc8-7063e100a963-snapshots\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.171959 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-ca-trust-extracted\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.171959 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171935    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vjvn\" (UniqueName: \"kubernetes.io/projected/3303870c-5cda-4ec0-b6df-6b54455ee748-kube-api-access-2vjvn\") pod \"managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w\" (UID: \"3303870c-5cda-4ec0-b6df-6b54455ee748\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w"
Jul 21 12:52:07.172063 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67ldz\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-kube-api-access-67ldz\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.172063 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.171990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7078810-2861-4a0f-a12c-16e80f3f2319-tmp\") pod \"klusterlet-addon-workmgr-5cd9b6688d-zrw92\" (UID: \"e7078810-2861-4a0f-a12c-16e80f3f2319\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"
Jul 21 12:52:07.172063 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/41f99d1e-551f-457e-8a74-87f9702496cf-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-b5v7d\" (UID: \"41f99d1e-551f-457e-8a74-87f9702496cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"
Jul 21 12:52:07.172063 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172053    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-bound-sa-token\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.172252 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z4wx\" (UniqueName: \"kubernetes.io/projected/d12b1d08-a810-4d9a-bcc8-7063e100a963-kube-api-access-6z4wx\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.172252 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172113    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-stats-auth\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.172252 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41f99d1e-551f-457e-8a74-87f9702496cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-b5v7d\" (UID: \"41f99d1e-551f-457e-8a74-87f9702496cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"
Jul 21 12:52:07.172252 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3303870c-5cda-4ec0-b6df-6b54455ee748-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w\" (UID: \"3303870c-5cda-4ec0-b6df-6b54455ee748\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w"
Jul 21 12:52:07.172252 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-registry-certificates\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.172252 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172243    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8f2f\" (UniqueName: \"kubernetes.io/projected/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-kube-api-access-g8f2f\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.172559 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172284    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-ca-trust-extracted\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.172720 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d12b1d08-a810-4d9a-bcc8-7063e100a963-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.172793 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172739    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d12b1d08-a810-4d9a-bcc8-7063e100a963-tmp\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.172793 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172720    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7078810-2861-4a0f-a12c-16e80f3f2319-tmp\") pod \"klusterlet-addon-workmgr-5cd9b6688d-zrw92\" (UID: \"e7078810-2861-4a0f-a12c-16e80f3f2319\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"
Jul 21 12:52:07.172793 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172764    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d12b1d08-a810-4d9a-bcc8-7063e100a963-serving-cert\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.172942 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d12b1d08-a810-4d9a-bcc8-7063e100a963-service-ca-bundle\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.172942 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172835    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-trusted-ca\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.172942 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.172860    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-service-ca-bundle\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.173198 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.173176    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-registry-certificates\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.173519 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.173495    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/41f99d1e-551f-457e-8a74-87f9702496cf-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-b5v7d\" (UID: \"41f99d1e-551f-457e-8a74-87f9702496cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"
Jul 21 12:52:07.174106 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.174055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-trusted-ca\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.174467 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.174446    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-installation-pull-secrets\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.174676 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.174476    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-registry-tls\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.174676 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.174598    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e7078810-2861-4a0f-a12c-16e80f3f2319-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cd9b6688d-zrw92\" (UID: \"e7078810-2861-4a0f-a12c-16e80f3f2319\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"
Jul 21 12:52:07.174814 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.174694    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-image-registry-private-configuration\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.175919 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.175896    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41f99d1e-551f-457e-8a74-87f9702496cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-b5v7d\" (UID: \"41f99d1e-551f-457e-8a74-87f9702496cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"
Jul 21 12:52:07.176335 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.176279    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3303870c-5cda-4ec0-b6df-6b54455ee748-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w\" (UID: \"3303870c-5cda-4ec0-b6df-6b54455ee748\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w"
Jul 21 12:52:07.180679 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.180645    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjd57\" (UniqueName: \"kubernetes.io/projected/9d55962c-a472-465a-8285-9c8bf0856ae4-kube-api-access-pjd57\") pod \"downloads-5bc67dcd46-d5ptb\" (UID: \"9d55962c-a472-465a-8285-9c8bf0856ae4\") " pod="openshift-console/downloads-5bc67dcd46-d5ptb"
Jul 21 12:52:07.182873 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.182847    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55sdr\" (UniqueName: \"kubernetes.io/projected/41f99d1e-551f-457e-8a74-87f9702496cf-kube-api-access-55sdr\") pod \"cluster-monitoring-operator-76cf8966f8-b5v7d\" (UID: \"41f99d1e-551f-457e-8a74-87f9702496cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"
Jul 21 12:52:07.183237 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.183212    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67ldz\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-kube-api-access-67ldz\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.183725 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.183696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vjvn\" (UniqueName: \"kubernetes.io/projected/3303870c-5cda-4ec0-b6df-6b54455ee748-kube-api-access-2vjvn\") pod \"managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w\" (UID: \"3303870c-5cda-4ec0-b6df-6b54455ee748\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w"
Jul 21 12:52:07.183820 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.183801    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-bound-sa-token\") pod \"image-registry-5b58b8bc7c-j8rpf\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") " pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.184780 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.184760    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crzcz\" (UniqueName: \"kubernetes.io/projected/e7078810-2861-4a0f-a12c-16e80f3f2319-kube-api-access-crzcz\") pod \"klusterlet-addon-workmgr-5cd9b6688d-zrw92\" (UID: \"e7078810-2861-4a0f-a12c-16e80f3f2319\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"
Jul 21 12:52:07.212372 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.212231    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:07.259217 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.259185    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.273917 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.273885    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:07.274065 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.274048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-default-certificate\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.274107 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.274095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-metrics-certs\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.274155 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.274142    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d12b1d08-a810-4d9a-bcc8-7063e100a963-snapshots\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.274188 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.274173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z4wx\" (UniqueName: \"kubernetes.io/projected/d12b1d08-a810-4d9a-bcc8-7063e100a963-kube-api-access-6z4wx\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.274230 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.274190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-stats-auth\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.274230 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.274223    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8f2f\" (UniqueName: \"kubernetes.io/projected/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-kube-api-access-g8f2f\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.274501 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.274476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d12b1d08-a810-4d9a-bcc8-7063e100a963-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.274604 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.274517    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d12b1d08-a810-4d9a-bcc8-7063e100a963-tmp\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.274604 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.274542    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d12b1d08-a810-4d9a-bcc8-7063e100a963-serving-cert\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.274604 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.274585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d12b1d08-a810-4d9a-bcc8-7063e100a963-service-ca-bundle\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.274769 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.274659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-service-ca-bundle\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.274875 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.274855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d12b1d08-a810-4d9a-bcc8-7063e100a963-tmp\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.275277 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.275253    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d12b1d08-a810-4d9a-bcc8-7063e100a963-service-ca-bundle\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.275277 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.275274    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-service-ca-bundle\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.275500 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.275482    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d12b1d08-a810-4d9a-bcc8-7063e100a963-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.275763 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.275741    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d12b1d08-a810-4d9a-bcc8-7063e100a963-snapshots\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.277123 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.277097    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-stats-auth\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.277195 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.277156    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-default-certificate\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.277324 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.277285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-metrics-certs\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.277777 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.277757    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d12b1d08-a810-4d9a-bcc8-7063e100a963-serving-cert\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.292571 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.292551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8f2f\" (UniqueName: \"kubernetes.io/projected/746485ca-3bb7-46e3-82b8-e9bdebf3ad96-kube-api-access-g8f2f\") pod \"router-default-785f46d866-4qbft\" (UID: \"746485ca-3bb7-46e3-82b8-e9bdebf3ad96\") " pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.293589 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.293570    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z4wx\" (UniqueName: \"kubernetes.io/projected/d12b1d08-a810-4d9a-bcc8-7063e100a963-kube-api-access-6z4wx\") pod \"insights-operator-555d9d7885-pnxvv\" (UID: \"d12b1d08-a810-4d9a-bcc8-7063e100a963\") " pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.294642 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.294620    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:07.311794 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.311767    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w"
Jul 21 12:52:07.322107 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.322078    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"
Jul 21 12:52:07.357628 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.357605    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"
Jul 21 12:52:07.370033 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.370002    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-d5ptb"
Jul 21 12:52:07.375655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.375632    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-trusted-ca\") pod \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") "
Jul 21 12:52:07.375795 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.375663    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-installation-pull-secrets\") pod \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") "
Jul 21 12:52:07.375795 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.375696    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-registry-tls\") pod \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") "
Jul 21 12:52:07.375795 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.375720    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67ldz\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-kube-api-access-67ldz\") pod \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") "
Jul 21 12:52:07.375795 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.375746    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-registry-certificates\") pod \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") "
Jul 21 12:52:07.375795 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.375772    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-ca-trust-extracted\") pod \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") "
Jul 21 12:52:07.375795 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.375793    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-image-registry-private-configuration\") pod \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") "
Jul 21 12:52:07.376074 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.375835    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-bound-sa-token\") pod \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\" (UID: \"ea2c78bd-ca63-45d3-bd9a-b610c7430ae2\") "
Jul 21 12:52:07.376690 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.376211    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2" (UID: "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:52:07.376690 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.376396    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2" (UID: "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:52:07.376690 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.376678    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-registry-certificates\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:07.376957 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.376723    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-ca-trust-extracted\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:07.377089 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.377061    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2" (UID: "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:52:07.379605 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.379441    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2" (UID: "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:52:07.379605 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.379539    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2" (UID: "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:52:07.379768 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.379723    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-kube-api-access-67ldz" (OuterVolumeSpecName: "kube-api-access-67ldz") pod "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2" (UID: "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2"). InnerVolumeSpecName "kube-api-access-67ldz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:52:07.380818 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.380524    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2" (UID: "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:52:07.384433 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.384395    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2" (UID: "ea2c78bd-ca63-45d3-bd9a-b610c7430ae2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:52:07.392979 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.392939    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:07.395988 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.395933    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-pnxvv"
Jul 21 12:52:07.412366 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.412313    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:07.478616 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.478446    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-image-registry-private-configuration\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:07.478616 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.478478    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-bound-sa-token\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:07.478616 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.478495    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-trusted-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:07.478616 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.478507    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-installation-pull-secrets\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:07.478616 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.478519    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-registry-tls\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:07.478616 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.478534    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67ldz\" (UniqueName: \"kubernetes.io/projected/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2-kube-api-access-67ldz\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:07.500141 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.500069    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w"]
Jul 21 12:52:07.506188 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:52:07.506137    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3303870c_5cda_4ec0_b6df_6b54455ee748.slice/crio-0f8da9770c494e8e913c53b097fcd602146c28026f1a3f0bf8104c0e70f136e8 WatchSource:0}: Error finding container 0f8da9770c494e8e913c53b097fcd602146c28026f1a3f0bf8104c0e70f136e8: Status 404 returned error can't find the container with id 0f8da9770c494e8e913c53b097fcd602146c28026f1a3f0bf8104c0e70f136e8
Jul 21 12:52:07.524441 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.524356    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"]
Jul 21 12:52:07.530617 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:52:07.530568    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7078810_2861_4a0f_a12c_16e80f3f2319.slice/crio-f037682a0f29ac84dce2feb7a237849fd38d1a1b2c990bfc1ec56ac6a425ecd0 WatchSource:0}: Error finding container f037682a0f29ac84dce2feb7a237849fd38d1a1b2c990bfc1ec56ac6a425ecd0: Status 404 returned error can't find the container with id f037682a0f29ac84dce2feb7a237849fd38d1a1b2c990bfc1ec56ac6a425ecd0
Jul 21 12:52:07.571344 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.571269    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d"]
Jul 21 12:52:07.587598 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.587567    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-d5ptb"]
Jul 21 12:52:07.594640 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:52:07.594608    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d55962c_a472_465a_8285_9c8bf0856ae4.slice/crio-b1754765cfea491e810e3d380201d14b9139f0c94d572573da680a52f5909e7a WatchSource:0}: Error finding container b1754765cfea491e810e3d380201d14b9139f0c94d572573da680a52f5909e7a: Status 404 returned error can't find the container with id b1754765cfea491e810e3d380201d14b9139f0c94d572573da680a52f5909e7a
Jul 21 12:52:07.609260 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.609223    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-pnxvv"]
Jul 21 12:52:07.612093 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:52:07.612061    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd12b1d08_a810_4d9a_bcc8_7063e100a963.slice/crio-31b39ab1b9530e6b0bb83c824c89f4578f539d6ea200882ddadd43a4d83ecb7d WatchSource:0}: Error finding container 31b39ab1b9530e6b0bb83c824c89f4578f539d6ea200882ddadd43a4d83ecb7d: Status 404 returned error can't find the container with id 31b39ab1b9530e6b0bb83c824c89f4578f539d6ea200882ddadd43a4d83ecb7d
Jul 21 12:52:07.626447 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.626419    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-785f46d866-4qbft"]
Jul 21 12:52:07.629059 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:52:07.629030    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod746485ca_3bb7_46e3_82b8_e9bdebf3ad96.slice/crio-ed86a2e839906d2fd0ccdf4d441c20e85a792b73f962fc935acbe56ab063283c WatchSource:0}: Error finding container ed86a2e839906d2fd0ccdf4d441c20e85a792b73f962fc935acbe56ab063283c: Status 404 returned error can't find the container with id ed86a2e839906d2fd0ccdf4d441c20e85a792b73f962fc935acbe56ab063283c
Jul 21 12:52:07.959670 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:07.959633    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:08.059337 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.059284    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:08.263801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.263702    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92" event={"ID":"e7078810-2861-4a0f-a12c-16e80f3f2319","Type":"ContainerStarted","Data":"f037682a0f29ac84dce2feb7a237849fd38d1a1b2c990bfc1ec56ac6a425ecd0"}
Jul 21 12:52:08.265098 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.265067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w" event={"ID":"3303870c-5cda-4ec0-b6df-6b54455ee748","Type":"ContainerStarted","Data":"0f8da9770c494e8e913c53b097fcd602146c28026f1a3f0bf8104c0e70f136e8"}
Jul 21 12:52:08.266793 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.266765    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-785f46d866-4qbft" event={"ID":"746485ca-3bb7-46e3-82b8-e9bdebf3ad96","Type":"ContainerStarted","Data":"6cc6db6a3a03353f0bf33cf519af28779d2d4d1dfdb979c91e3f562a01f37682"}
Jul 21 12:52:08.266925 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.266813    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-785f46d866-4qbft" event={"ID":"746485ca-3bb7-46e3-82b8-e9bdebf3ad96","Type":"ContainerStarted","Data":"ed86a2e839906d2fd0ccdf4d441c20e85a792b73f962fc935acbe56ab063283c"}
Jul 21 12:52:08.268062 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.268032    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-pnxvv" event={"ID":"d12b1d08-a810-4d9a-bcc8-7063e100a963","Type":"ContainerStarted","Data":"31b39ab1b9530e6b0bb83c824c89f4578f539d6ea200882ddadd43a4d83ecb7d"}
Jul 21 12:52:08.269442 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.269403    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-d5ptb" event={"ID":"9d55962c-a472-465a-8285-9c8bf0856ae4","Type":"ContainerStarted","Data":"b1754765cfea491e810e3d380201d14b9139f0c94d572573da680a52f5909e7a"}
Jul 21 12:52:08.270821 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.270789    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d" event={"ID":"41f99d1e-551f-457e-8a74-87f9702496cf","Type":"ContainerStarted","Data":"6910e12a1099797f45309fd419c773d85fe8f1ffb75909a8a70f17affb289313"}
Jul 21 12:52:08.270934 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.270820    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"
Jul 21 12:52:08.320644 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.320567    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-785f46d866-4qbft" podStartSLOduration=1.320546924 podStartE2EDuration="1.320546924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:52:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:08.292901325 +0000 UTC m=+60.818404482" watchObservedRunningTime="2026-07-21 12:52:08.320546924 +0000 UTC m=+60.846050080"
Jul 21 12:52:08.331316 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.331262    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"]
Jul 21 12:52:08.342768 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.342732    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"]
Jul 21 12:52:08.342768 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.342768    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5b58b8bc7c-j8rpf"]
Jul 21 12:52:08.342993 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.342866    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.345511 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.345486    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 21 12:52:08.345734 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.345666    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jjwbz\""
Jul 21 12:52:08.345899 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.345781    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 21 12:52:08.346021 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.346009    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 21 12:52:08.355577 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.353638    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 21 12:52:08.356908 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.356881    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"]
Jul 21 12:52:08.370477 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.370441    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:08.386789 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.386749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c5394394-40bb-4611-b59b-40721647c330-installation-pull-secrets\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.386963 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.386802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c5394394-40bb-4611-b59b-40721647c330-ca-trust-extracted\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.386963 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.386837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c5394394-40bb-4611-b59b-40721647c330-registry-certificates\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.386963 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.386924    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c5394394-40bb-4611-b59b-40721647c330-image-registry-private-configuration\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.387094 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.386969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5394394-40bb-4611-b59b-40721647c330-trusted-ca\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.387094 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.386991    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5394394-40bb-4611-b59b-40721647c330-registry-tls\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.387094 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.387010    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c5394394-40bb-4611-b59b-40721647c330-bound-sa-token\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.387094 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.387035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gptmw\" (UniqueName: \"kubernetes.io/projected/c5394394-40bb-4611-b59b-40721647c330-kube-api-access-gptmw\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.413750 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.413712    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:08.416167 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.416145    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:08.435609 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.435573    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:08.488306 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.488261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c5394394-40bb-4611-b59b-40721647c330-installation-pull-secrets\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.488468 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.488341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c5394394-40bb-4611-b59b-40721647c330-ca-trust-extracted\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.488468 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.488380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c5394394-40bb-4611-b59b-40721647c330-registry-certificates\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.488468 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.488421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c5394394-40bb-4611-b59b-40721647c330-image-registry-private-configuration\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.488468 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.488457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5394394-40bb-4611-b59b-40721647c330-trusted-ca\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.488675 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.488484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5394394-40bb-4611-b59b-40721647c330-registry-tls\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.488675 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.488504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c5394394-40bb-4611-b59b-40721647c330-bound-sa-token\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.488675 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.488528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gptmw\" (UniqueName: \"kubernetes.io/projected/c5394394-40bb-4611-b59b-40721647c330-kube-api-access-gptmw\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.488884 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.488857    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c5394394-40bb-4611-b59b-40721647c330-ca-trust-extracted\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.489965 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.489933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5394394-40bb-4611-b59b-40721647c330-trusted-ca\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.490314 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.490188    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c5394394-40bb-4611-b59b-40721647c330-registry-certificates\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.492181 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.492158    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5394394-40bb-4611-b59b-40721647c330-registry-tls\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.498035 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.498010    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gptmw\" (UniqueName: \"kubernetes.io/projected/c5394394-40bb-4611-b59b-40721647c330-kube-api-access-gptmw\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.499536 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.499509    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c5394394-40bb-4611-b59b-40721647c330-bound-sa-token\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.502464 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.502431    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c5394394-40bb-4611-b59b-40721647c330-installation-pull-secrets\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.505952 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.505929    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c5394394-40bb-4611-b59b-40721647c330-image-registry-private-configuration\") pod \"image-registry-6bbc7f78d5-cfgw7\" (UID: \"c5394394-40bb-4611-b59b-40721647c330\") " pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.654179 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.654130    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:08.841082 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:08.841040    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"]
Jul 21 12:52:08.847760 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:52:08.847729    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5394394_40bb_4611_b59b_40721647c330.slice/crio-af589ddb279185a90c36d984eec1cea2af96c841ad292b341db9e3dd745ea53d WatchSource:0}: Error finding container af589ddb279185a90c36d984eec1cea2af96c841ad292b341db9e3dd745ea53d: Status 404 returned error can't find the container with id af589ddb279185a90c36d984eec1cea2af96c841ad292b341db9e3dd745ea53d
Jul 21 12:52:09.282976 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:09.281345    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7" event={"ID":"c5394394-40bb-4611-b59b-40721647c330","Type":"ContainerStarted","Data":"272994d6a7f1ed05e742a19b340ed7765a98721512d1dcbe6e53687259d64e70"}
Jul 21 12:52:09.282976 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:09.281409    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7" event={"ID":"c5394394-40bb-4611-b59b-40721647c330","Type":"ContainerStarted","Data":"af589ddb279185a90c36d984eec1cea2af96c841ad292b341db9e3dd745ea53d"}
Jul 21 12:52:09.282976 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:09.281440    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:09.283539 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:09.281489    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:09.283887 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:09.283837    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-785f46d866-4qbft"
Jul 21 12:52:09.309093 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:09.309030    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7" podStartSLOduration=3.309006459 podStartE2EDuration="3.309006459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:52:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:09.307287338 +0000 UTC m=+61.832790495" watchObservedRunningTime="2026-07-21 12:52:09.309006459 +0000 UTC m=+61.834509617"
Jul 21 12:52:09.997083 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:09.997044    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea2c78bd-ca63-45d3-bd9a-b610c7430ae2" path="/var/lib/kubelet/pods/ea2c78bd-ca63-45d3-bd9a-b610c7430ae2/volumes"
Jul 21 12:52:10.433949 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.432618    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78849cc87c-6dbtr"]
Jul 21 12:52:10.436930 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.435474    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.439492 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.439468    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 21 12:52:10.439874 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.439857    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 21 12:52:10.440361 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.440128    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9l4m2\""
Jul 21 12:52:10.440361 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.440346    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 21 12:52:10.441633 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.440570    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 21 12:52:10.441633 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.440832    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 21 12:52:10.446319 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.445206    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78849cc87c-6dbtr"]
Jul 21 12:52:10.510773 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.510731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-config\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.511022 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.510843    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-serving-cert\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.511022 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.510891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-service-ca\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.511022 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.510930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-oauth-config\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.511022 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.510953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-oauth-serving-cert\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.511022 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.510983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mv4p\" (UniqueName: \"kubernetes.io/projected/42b85580-cbe8-42da-a5b2-7d748e31cfc5-kube-api-access-2mv4p\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.612179 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.612080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-serving-cert\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.612179 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.612163    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-service-ca\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.612444 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.612216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-oauth-config\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.612444 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.612237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-oauth-serving-cert\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.612444 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.612256    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mv4p\" (UniqueName: \"kubernetes.io/projected/42b85580-cbe8-42da-a5b2-7d748e31cfc5-kube-api-access-2mv4p\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.612444 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.612306    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-config\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.613600 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.613563    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-oauth-serving-cert\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.614087 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.614056    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-service-ca\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.614203 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.614139    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-config\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.616195 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.616172    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-oauth-config\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.618795 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.618699    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-serving-cert\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.629186 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.629158    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mv4p\" (UniqueName: \"kubernetes.io/projected/42b85580-cbe8-42da-a5b2-7d748e31cfc5-kube-api-access-2mv4p\") pod \"console-78849cc87c-6dbtr\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") " pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:10.759968 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:10.759557    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:11.418272 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:11.418238    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:11.482161 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:11.482125    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:11.539003 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:11.538962    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:11.630425 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:11.630385    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:11.695326 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:11.695211    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:12.248800 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:12.248766    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rtn7b"
Jul 21 12:52:13.036849 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.036707    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:13.091414 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.091357    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:13.382280 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.382124    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78849cc87c-6dbtr"]
Jul 21 12:52:13.384840 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:52:13.384800    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42b85580_cbe8_42da_a5b2_7d748e31cfc5.slice/crio-ecf09a9aedbb63e8ed228c6f89d76128c1ea86e105fba4f0ecfda0a7a0ecff20 WatchSource:0}: Error finding container ecf09a9aedbb63e8ed228c6f89d76128c1ea86e105fba4f0ecfda0a7a0ecff20: Status 404 returned error can't find the container with id ecf09a9aedbb63e8ed228c6f89d76128c1ea86e105fba4f0ecfda0a7a0ecff20
Jul 21 12:52:13.616697 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.616619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:13.719312 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.719197    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:13.752263 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.751394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:52:13.752263 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.751497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs\") pod \"network-metrics-daemon-rjmvf\" (UID: \"e4069bb1-2913-45bf-ade6-79ab04c94fa8\") " pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:52:13.752263 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.751528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5sxv\" (UniqueName: \"kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv\") pod \"network-check-target-kpnng\" (UID: \"1f77f91a-1189-4c6c-b6cf-d5213e58ec43\") " pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:52:13.754935 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.754685    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 12:52:13.755775 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.755399    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 12:52:13.755775 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.755622    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 12:52:13.765412 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.765198    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 12:52:13.766325 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.766283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4069bb1-2913-45bf-ade6-79ab04c94fa8-metrics-certs\") pod \"network-metrics-daemon-rjmvf\" (UID: \"e4069bb1-2913-45bf-ade6-79ab04c94fa8\") " pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:52:13.767922 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.767877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49c51a2b-86c0-4f43-af87-0f9dc3a56986-original-pull-secret\") pod \"global-pull-secret-syncer-n7pwd\" (UID: \"49c51a2b-86c0-4f43-af87-0f9dc3a56986\") " pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:52:13.778439 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.778382    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5sxv\" (UniqueName: \"kubernetes.io/projected/1f77f91a-1189-4c6c-b6cf-d5213e58ec43-kube-api-access-m5sxv\") pod \"network-check-target-kpnng\" (UID: \"1f77f91a-1189-4c6c-b6cf-d5213e58ec43\") " pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:52:13.803436 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.803018    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:13.914362 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.913779    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7pwd"
Jul 21 12:52:13.916534 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.916328    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5ccp9\""
Jul 21 12:52:13.924738 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.924193    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:52:13.924738 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.924508    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rtsf5\""
Jul 21 12:52:13.935388 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:13.935284    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjmvf"
Jul 21 12:52:14.107437 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.107241    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:14.148433 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.147621    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kv8z4"]
Jul 21 12:52:14.159084 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.157353    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.173388 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.162139    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lzpc5\""
Jul 21 12:52:14.173388 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.162404    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 21 12:52:14.173388 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.162569    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 21 12:52:14.173388 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.163652    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kv8z4"]
Jul 21 12:52:14.175435 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.175368    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n7pwd"]
Jul 21 12:52:14.189180 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:52:14.189148    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49c51a2b_86c0_4f43_af87_0f9dc3a56986.slice/crio-f05e6e990426d1feb8338868dcab683b7f4c5fdb6ff1089974c0ea8f7a55f839 WatchSource:0}: Error finding container f05e6e990426d1feb8338868dcab683b7f4c5fdb6ff1089974c0ea8f7a55f839: Status 404 returned error can't find the container with id f05e6e990426d1feb8338868dcab683b7f4c5fdb6ff1089974c0ea8f7a55f839
Jul 21 12:52:14.196528 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.196441    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rjmvf"]
Jul 21 12:52:14.210447 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.203133    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:14.210447 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.205759    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kpnng"]
Jul 21 12:52:14.230933 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:52:14.230897    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f77f91a_1189_4c6c_b6cf_d5213e58ec43.slice/crio-bde8463ce921abb0c0813e3f60bde6adbbb47481157fee0e80e9785ad8cf5a0d WatchSource:0}: Error finding container bde8463ce921abb0c0813e3f60bde6adbbb47481157fee0e80e9785ad8cf5a0d: Status 404 returned error can't find the container with id bde8463ce921abb0c0813e3f60bde6adbbb47481157fee0e80e9785ad8cf5a0d
Jul 21 12:52:14.256444 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.256405    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/66a4327a-f8c0-4865-834d-f964c9b00a75-data-volume\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.256626 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.256468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.256948 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.256739    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.256948 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.256831    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xht9\" (UniqueName: \"kubernetes.io/projected/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-api-access-9xht9\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.256948 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.256860    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/66a4327a-f8c0-4865-834d-f964c9b00a75-crio-socket\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.320024 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.319966    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:14.320815 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.320700    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rjmvf" event={"ID":"e4069bb1-2913-45bf-ade6-79ab04c94fa8","Type":"ContainerStarted","Data":"5be2d1f850bcffd940a709da1474d3b27554bc53debce172c21c25e4e3c87197"}
Jul 21 12:52:14.323687 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.323662    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78849cc87c-6dbtr" event={"ID":"42b85580-cbe8-42da-a5b2-7d748e31cfc5","Type":"ContainerStarted","Data":"ecf09a9aedbb63e8ed228c6f89d76128c1ea86e105fba4f0ecfda0a7a0ecff20"}
Jul 21 12:52:14.326186 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.326160    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-pnxvv" event={"ID":"d12b1d08-a810-4d9a-bcc8-7063e100a963","Type":"ContainerStarted","Data":"e23d098e05a72fb3514a6b06633dffd51aee2b0589d145497859b703ecd00ee8"}
Jul 21 12:52:14.328979 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.328948    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d" event={"ID":"41f99d1e-551f-457e-8a74-87f9702496cf","Type":"ContainerStarted","Data":"17fa650a85e41127308988230ea9fc0932865a78ad615e41158d298a488bdf5b"}
Jul 21 12:52:14.330523 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.330501    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kpnng" event={"ID":"1f77f91a-1189-4c6c-b6cf-d5213e58ec43","Type":"ContainerStarted","Data":"bde8463ce921abb0c0813e3f60bde6adbbb47481157fee0e80e9785ad8cf5a0d"}
Jul 21 12:52:14.332347 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.332325    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92" event={"ID":"e7078810-2861-4a0f-a12c-16e80f3f2319","Type":"ContainerStarted","Data":"ca637828a858fef4e0665299c7dfecba54c2dab1ebaf5804c600fa0f0b1059bf"}
Jul 21 12:52:14.333000 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.332952    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"
Jul 21 12:52:14.334356 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.334277    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n7pwd" event={"ID":"49c51a2b-86c0-4f43-af87-0f9dc3a56986","Type":"ContainerStarted","Data":"f05e6e990426d1feb8338868dcab683b7f4c5fdb6ff1089974c0ea8f7a55f839"}
Jul 21 12:52:14.335009 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.334991    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92"
Jul 21 12:52:14.343809 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.343758    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-pnxvv" podStartSLOduration=1.754246349 podStartE2EDuration="7.34374598s" totalImagesPullingTime="5.589499631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:52:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:14.34349322 +0000 UTC m=+66.868996376" watchObservedRunningTime="2026-07-21 12:52:14.34374598 +0000 UTC m=+66.869249134"
Jul 21 12:52:14.357967 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.357916    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xht9\" (UniqueName: \"kubernetes.io/projected/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-api-access-9xht9\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.357967 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.357962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/66a4327a-f8c0-4865-834d-f964c9b00a75-crio-socket\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.358172 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.358008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/66a4327a-f8c0-4865-834d-f964c9b00a75-data-volume\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.358172 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.358098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.358275 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.358194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.358475 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:14.358457    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:52:14.358537 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:14.358529    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls podName:66a4327a-f8c0-4865-834d-f964c9b00a75 nodeName:}" failed. No retries permitted until 2026-07-21 12:52:14.858508125 +0000 UTC m=+67.384011274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kv8z4" (UID: "66a4327a-f8c0-4865-834d-f964c9b00a75") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:52:14.358849 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.358829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/66a4327a-f8c0-4865-834d-f964c9b00a75-crio-socket\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.359179 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.359138    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/66a4327a-f8c0-4865-834d-f964c9b00a75-data-volume\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.359947 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:14.359905    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:52:14.360047 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:14.359971    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm podName:66a4327a-f8c0-4865-834d-f964c9b00a75 nodeName:}" failed. No retries permitted until 2026-07-21 12:52:14.859956168 +0000 UTC m=+67.385459308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kv8z4" (UID: "66a4327a-f8c0-4865-834d-f964c9b00a75") : configmap "kube-rbac-proxy" not found
Jul 21 12:52:14.369456 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.369428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xht9\" (UniqueName: \"kubernetes.io/projected/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-api-access-9xht9\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.382432 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.382372    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-b5v7d" podStartSLOduration=1.757802211 podStartE2EDuration="7.382347641s" totalImagesPullingTime="5.62454543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:52:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:14.364023154 +0000 UTC m=+66.889526321" watchObservedRunningTime="2026-07-21 12:52:14.382347641 +0000 UTC m=+66.907850782"
Jul 21 12:52:14.383177 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.383133    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd9b6688d-zrw92" podStartSLOduration=2.689956377 podStartE2EDuration="8.383115698s" totalImagesPullingTime="5.693159321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:52:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:14.381673878 +0000 UTC m=+66.907177038" watchObservedRunningTime="2026-07-21 12:52:14.383115698 +0000 UTC m=+66.908618854"
Jul 21 12:52:14.864447 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.864338    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.864447 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:14.864426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:14.864785 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:14.864597    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:52:14.864785 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:14.864705    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls podName:66a4327a-f8c0-4865-834d-f964c9b00a75 nodeName:}" failed. No retries permitted until 2026-07-21 12:52:15.864688548 +0000 UTC m=+68.390191685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kv8z4" (UID: "66a4327a-f8c0-4865-834d-f964c9b00a75") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:52:14.864785 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:14.864780    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:52:14.864920 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:14.864815    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm podName:66a4327a-f8c0-4865-834d-f964c9b00a75 nodeName:}" failed. No retries permitted until 2026-07-21 12:52:15.86480363 +0000 UTC m=+68.390306777 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kv8z4" (UID: "66a4327a-f8c0-4865-834d-f964c9b00a75") : configmap "kube-rbac-proxy" not found
Jul 21 12:52:15.613008 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:15.612979    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rtn7b_698e2686-c7ef-4e97-bed4-7cc24e1bd4fe/dns/0.log"
Jul 21 12:52:15.815917 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:15.815889    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rtn7b_698e2686-c7ef-4e97-bed4-7cc24e1bd4fe/kube-rbac-proxy/0.log"
Jul 21 12:52:15.875052 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:15.874966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:15.875052 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:15.875044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:15.875259 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:15.875241    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:52:15.875371 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:15.875339    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls podName:66a4327a-f8c0-4865-834d-f964c9b00a75 nodeName:}" failed. No retries permitted until 2026-07-21 12:52:17.875319431 +0000 UTC m=+70.400822582 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kv8z4" (UID: "66a4327a-f8c0-4865-834d-f964c9b00a75") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:52:15.875749 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:15.875733    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:52:15.875798 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:15.875788    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm podName:66a4327a-f8c0-4865-834d-f964c9b00a75 nodeName:}" failed. No retries permitted until 2026-07-21 12:52:17.875772045 +0000 UTC m=+70.401275197 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kv8z4" (UID: "66a4327a-f8c0-4865-834d-f964c9b00a75") : configmap "kube-rbac-proxy" not found
Jul 21 12:52:16.023212 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:16.023159    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:16.091317 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:16.091236    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:16.350886 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:16.350849    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w" event={"ID":"3303870c-5cda-4ec0-b6df-6b54455ee748","Type":"ContainerStarted","Data":"22c3dfe54f16bc12bdb9e81e21be525bcc6de3c43f8ca9c7fa7ab5cef3bfd933"}
Jul 21 12:52:16.366642 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:16.366591    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78cd9dfcc8-cwx2w" podStartSLOduration=2.540374448 podStartE2EDuration="10.366570513s" totalImagesPullingTime="7.826196065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:52:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:16.365808016 +0000 UTC m=+68.891311174" watchObservedRunningTime="2026-07-21 12:52:16.366570513 +0000 UTC m=+68.892073671"
Jul 21 12:52:16.414050 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:16.414020    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w92wm_fcebc074-dea4-4fe1-9208-f01443f8720b/dns-node-resolver/0.log"
Jul 21 12:52:16.613043 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:16.612968    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6bbc7f78d5-cfgw7_c5394394-40bb-4611-b59b-40721647c330/registry/0.log"
Jul 21 12:52:16.736144 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:16.736107    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:16.833456 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:16.833401    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:17.416175 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:17.415961    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nln7n_5bc0b65d-775f-43f5-b762-26ae86e132d1/node-ca/0.log"
Jul 21 12:52:17.617537 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:17.617506    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-785f46d866-4qbft_746485ca-3bb7-46e3-82b8-e9bdebf3ad96/router/0.log"
Jul 21 12:52:17.817974 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:17.817234    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b6h6z_2726d88e-bdde-4182-a2a5-2a1224965db8/serve-healthcheck-canary/0.log"
Jul 21 12:52:17.844387 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:17.844346    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:17.895985 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:17.895936    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:17.896176 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:17.896011    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:17.896287 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:17.896264    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:52:17.896379 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:17.896357    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:52:17.896439 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:17.896366    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls podName:66a4327a-f8c0-4865-834d-f964c9b00a75 nodeName:}" failed. No retries permitted until 2026-07-21 12:52:21.896345821 +0000 UTC m=+74.421848968 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kv8z4" (UID: "66a4327a-f8c0-4865-834d-f964c9b00a75") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:52:17.896500 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:17.896444    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm podName:66a4327a-f8c0-4865-834d-f964c9b00a75 nodeName:}" failed. No retries permitted until 2026-07-21 12:52:21.896422624 +0000 UTC m=+74.421925769 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kv8z4" (UID: "66a4327a-f8c0-4865-834d-f964c9b00a75") : configmap "kube-rbac-proxy" not found
Jul 21 12:52:17.945219 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:17.945149    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:19.458449 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:19.458373    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:20.230594 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:20.230553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:20.748533 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:20.748495    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:20.810509 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:20.810483    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:20.862162 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:20.862126    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:21.379782 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:21.379747    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kpnng" event={"ID":"1f77f91a-1189-4c6c-b6cf-d5213e58ec43","Type":"ContainerStarted","Data":"21c9daa0609f9fd2d1f407ea6ae5af710caf8462b967e25bf3eb65e19aca8a07"}
Jul 21 12:52:21.379962 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:21.379863    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:52:21.382082 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:21.381749    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n7pwd" event={"ID":"49c51a2b-86c0-4f43-af87-0f9dc3a56986","Type":"ContainerStarted","Data":"17b7d7cc863b1e4a204032c9e4c5f1faa5e9f39e632a33a92f082ecd57401d37"}
Jul 21 12:52:21.383566 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:21.383541    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rjmvf" event={"ID":"e4069bb1-2913-45bf-ade6-79ab04c94fa8","Type":"ContainerStarted","Data":"c911d12546d046cf8594e7f4e622fa6393cc6bb919e11fca8f5b4eea769f57f5"}
Jul 21 12:52:21.385215 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:21.385190    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78849cc87c-6dbtr" event={"ID":"42b85580-cbe8-42da-a5b2-7d748e31cfc5","Type":"ContainerStarted","Data":"3de5fe4249d3a978ea1bad2911c496ad5aec05490bafba2bc8fa720992e5b195"}
Jul 21 12:52:21.396844 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:21.396796    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kpnng" podStartSLOduration=66.646970066 podStartE2EDuration="1m13.396780124s" totalImagesPullingTime="6.749810058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:21.39563095 +0000 UTC m=+73.921134123" watchObservedRunningTime="2026-07-21 12:52:21.396780124 +0000 UTC m=+73.922283279"
Jul 21 12:52:21.414636 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:21.414580    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78849cc87c-6dbtr" podStartSLOduration=3.823226793 podStartE2EDuration="11.414560141s" totalImagesPullingTime="7.591333348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:52:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:21.412807675 +0000 UTC m=+73.938310832" watchObservedRunningTime="2026-07-21 12:52:21.414560141 +0000 UTC m=+73.940063298"
Jul 21 12:52:21.429683 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:21.429581    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:21.429835 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:21.429733    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n7pwd" podStartSLOduration=66.624770955 podStartE2EDuration="1m13.429718772s" totalImagesPullingTime="6.804947817s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:21.427789475 +0000 UTC m=+73.953292630" watchObservedRunningTime="2026-07-21 12:52:21.429718772 +0000 UTC m=+73.955221928"
Jul 21 12:52:21.529184 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:21.529134    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:21.939108 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:21.939069    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:21.939578 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:21.939125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:21.939578 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:21.939209    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:52:21.939578 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:21.939238    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:52:21.939578 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:21.939340    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm podName:66a4327a-f8c0-4865-834d-f964c9b00a75 nodeName:}" failed. No retries permitted until 2026-07-21 12:52:29.939316825 +0000 UTC m=+82.464819978 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kv8z4" (UID: "66a4327a-f8c0-4865-834d-f964c9b00a75") : configmap "kube-rbac-proxy" not found
Jul 21 12:52:21.939578 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:21.939365    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls podName:66a4327a-f8c0-4865-834d-f964c9b00a75 nodeName:}" failed. No retries permitted until 2026-07-21 12:52:29.939354575 +0000 UTC m=+82.464857722 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kv8z4" (UID: "66a4327a-f8c0-4865-834d-f964c9b00a75") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:52:22.016401 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:22.016360    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:22.117409 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:22.117372    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:22.271863 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:22.271773    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:23.974770 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:23.974724    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:24.078566 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.078530    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:24.225999 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.225890    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:24.283202 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.283163    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hbqlt"]
Jul 21 12:52:24.288550 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.287942    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.290844 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.290821    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 21 12:52:24.291441 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.291422    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fjmwx\""
Jul 21 12:52:24.291538 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.291476    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 21 12:52:24.291579 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.291420    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 21 12:52:24.292616 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.292596    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 21 12:52:24.344388 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.344356    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:24.360509 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.360470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9grg9\" (UniqueName: \"kubernetes.io/projected/66096f3e-3cfa-48e0-ad97-0104d69436ea-kube-api-access-9grg9\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.360696 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.360532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-tls\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.360696 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.360625    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66096f3e-3cfa-48e0-ad97-0104d69436ea-sys\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.360696 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.360673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-wtmp\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.360914 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.360706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-accelerators-collector-config\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.360914 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.360816    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-textfile\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.360914 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.360849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/66096f3e-3cfa-48e0-ad97-0104d69436ea-root\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.360914 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.360876    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66096f3e-3cfa-48e0-ad97-0104d69436ea-metrics-client-ca\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.361154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.360954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.462038 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.461995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-tls\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.462229 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.462068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66096f3e-3cfa-48e0-ad97-0104d69436ea-sys\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.462229 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.462099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-wtmp\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.462229 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.462125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-accelerators-collector-config\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.462229 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.462189    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-textfile\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.462229 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.462217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/66096f3e-3cfa-48e0-ad97-0104d69436ea-root\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.462504 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.462233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66096f3e-3cfa-48e0-ad97-0104d69436ea-metrics-client-ca\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.462504 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.462269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.462504 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.462318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9grg9\" (UniqueName: \"kubernetes.io/projected/66096f3e-3cfa-48e0-ad97-0104d69436ea-kube-api-access-9grg9\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.462781 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:24.462761    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 21 12:52:24.462843 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:24.462832    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-tls podName:66096f3e-3cfa-48e0-ad97-0104d69436ea nodeName:}" failed. No retries permitted until 2026-07-21 12:52:24.962811149 +0000 UTC m=+77.488314301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-tls") pod "node-exporter-hbqlt" (UID: "66096f3e-3cfa-48e0-ad97-0104d69436ea") : secret "node-exporter-tls" not found
Jul 21 12:52:24.463091 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.463072    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66096f3e-3cfa-48e0-ad97-0104d69436ea-sys\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.463280 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.463260    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/66096f3e-3cfa-48e0-ad97-0104d69436ea-root\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.463841 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.463821    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66096f3e-3cfa-48e0-ad97-0104d69436ea-metrics-client-ca\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.463905 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.463821    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-accelerators-collector-config\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.464063 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.464044    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-textfile\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.464148 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.464129    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-wtmp\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.466783 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.466761    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.473198 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.473168    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9grg9\" (UniqueName: \"kubernetes.io/projected/66096f3e-3cfa-48e0-ad97-0104d69436ea-kube-api-access-9grg9\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.499950 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.499837    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:24.600266 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.600175    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:24.782178 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.782066    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:24.882905 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.882868    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:24.966819 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.966783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-tls\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.969785 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.969754    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/66096f3e-3cfa-48e0-ad97-0104d69436ea-node-exporter-tls\") pod \"node-exporter-hbqlt\" (UID: \"66096f3e-3cfa-48e0-ad97-0104d69436ea\") " pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:24.983029 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:24.983001    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:25.115359 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:25.115322    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:25.201230 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:25.201196    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hbqlt"
Jul 21 12:52:25.308774 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:25.308708    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:25.381627 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:25.381523    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:26.262267 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:26.262226    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:26.726584 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:26.726548    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:27.171631 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:27.171555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:27.272163 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:27.272128    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:27.414982 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:27.414944    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:28.042531 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:28.042487    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:28.143515 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:28.143387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:28.246020 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:28.245982    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:28.486555 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:52:28.486524    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66096f3e_3cfa_48e0_ad97_0104d69436ea.slice/crio-f0cef80a519516f9bc0287cf1b14cea165cf45ed53627cdceaa258b43d56d406 WatchSource:0}: Error finding container f0cef80a519516f9bc0287cf1b14cea165cf45ed53627cdceaa258b43d56d406: Status 404 returned error can't find the container with id f0cef80a519516f9bc0287cf1b14cea165cf45ed53627cdceaa258b43d56d406
Jul 21 12:52:29.025185 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.025151    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:29.184228 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.184125    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:29.285480 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.285429    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:29.418972 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.418913    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rjmvf" event={"ID":"e4069bb1-2913-45bf-ade6-79ab04c94fa8","Type":"ContainerStarted","Data":"b816fa0aa83a454b14e78cc0136707b0aa0e2504a34f6052ea34512eae9e2c10"}
Jul 21 12:52:29.422402 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.422041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-d5ptb" event={"ID":"9d55962c-a472-465a-8285-9c8bf0856ae4","Type":"ContainerStarted","Data":"33b5a62205d596b3baf02c98883bf5671376bd1786fbef36bc1be4e1967b8073"}
Jul 21 12:52:29.422402 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.422343    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-d5ptb"
Jul 21 12:52:29.424141 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.424112    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hbqlt" event={"ID":"66096f3e-3cfa-48e0-ad97-0104d69436ea","Type":"ContainerStarted","Data":"f0cef80a519516f9bc0287cf1b14cea165cf45ed53627cdceaa258b43d56d406"}
Jul 21 12:52:29.437206 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.437096    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rjmvf" podStartSLOduration=74.689521625 podStartE2EDuration="1m21.437076996s" totalImagesPullingTime="6.747555371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:29.436445419 +0000 UTC m=+81.961948599" watchObservedRunningTime="2026-07-21 12:52:29.437076996 +0000 UTC m=+81.962580154"
Jul 21 12:52:29.440810 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.440765    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-d5ptb"
Jul 21 12:52:29.453862 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.453701    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-d5ptb" podStartSLOduration=1.178597261 podStartE2EDuration="22.453656954s" totalImagesPullingTime="21.275059693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:52:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:29.452250191 +0000 UTC m=+81.977753345" watchObservedRunningTime="2026-07-21 12:52:29.453656954 +0000 UTC m=+81.979160118"
Jul 21 12:52:29.562436 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.562403    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:29.633553 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.633465    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:29.799649 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.799570    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78849cc87c-6dbtr"]
Jul 21 12:52:29.866524 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.866485    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:29.929538 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:29.929501    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:30.012110 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.012066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:30.012326 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.012142    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:30.012752 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.012721    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66a4327a-f8c0-4865-834d-f964c9b00a75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:30.014948 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.014922    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66a4327a-f8c0-4865-834d-f964c9b00a75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kv8z4\" (UID: \"66a4327a-f8c0-4865-834d-f964c9b00a75\") " pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:30.043177 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.043138    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:30.074352 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.074239    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kv8z4"
Jul 21 12:52:30.285119 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.285092    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kv8z4"]
Jul 21 12:52:30.292996 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:52:30.292871    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66a4327a_f8c0_4865_834d_f964c9b00a75.slice/crio-b75a2d2aa07bdd984ecea7d5a93a593088d35bcf7b4285c5772725631a83d70c WatchSource:0}: Error finding container b75a2d2aa07bdd984ecea7d5a93a593088d35bcf7b4285c5772725631a83d70c: Status 404 returned error can't find the container with id b75a2d2aa07bdd984ecea7d5a93a593088d35bcf7b4285c5772725631a83d70c
Jul 21 12:52:30.430637 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.430577    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kv8z4" event={"ID":"66a4327a-f8c0-4865-834d-f964c9b00a75","Type":"ContainerStarted","Data":"b75a2d2aa07bdd984ecea7d5a93a593088d35bcf7b4285c5772725631a83d70c"}
Jul 21 12:52:30.433394 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.432530    2563 generic.go:356] "Generic (PLEG): container finished" podID="66096f3e-3cfa-48e0-ad97-0104d69436ea" containerID="0c7f105b24690c1176ebaa68576e1e10912d087a43a3e8dcbd6b095aeca9d47e" exitCode=0
Jul 21 12:52:30.433394 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.432648    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hbqlt" event={"ID":"66096f3e-3cfa-48e0-ad97-0104d69436ea","Type":"ContainerDied","Data":"0c7f105b24690c1176ebaa68576e1e10912d087a43a3e8dcbd6b095aeca9d47e"}
Jul 21 12:52:30.455217 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.455103    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:30.539605 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.539570    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:30.608476 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.608441    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:30.760996 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:30.760957    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:31.296130 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:31.296046    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6bbc7f78d5-cfgw7"
Jul 21 12:52:31.442142 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:31.442032    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hbqlt" event={"ID":"66096f3e-3cfa-48e0-ad97-0104d69436ea","Type":"ContainerStarted","Data":"589eafb34bb44e19067ddd55dc2cd52cf2287632c6e534cef27a141591ce274c"}
Jul 21 12:52:31.442142 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:31.442091    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hbqlt" event={"ID":"66096f3e-3cfa-48e0-ad97-0104d69436ea","Type":"ContainerStarted","Data":"450417bf748024e8b3537d4e7528ac2ea8a250d1193ba0a535800d1ef9eb5925"}
Jul 21 12:52:31.470334 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:31.470261    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hbqlt" podStartSLOduration=5.753118203 podStartE2EDuration="7.470236662s" totalImagesPullingTime="1.614988288s" totalInitContainerRuntime="102.130171ms" isStatefulPod=true podCreationTimestamp="2026-07-21 12:52:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:31.469146368 +0000 UTC m=+83.994649525" watchObservedRunningTime="2026-07-21 12:52:31.470236662 +0000 UTC m=+83.995739818"
Jul 21 12:52:31.499977 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:31.499735    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:31.610126 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:31.610091    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:31.701048 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:31.700964    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:31.802115 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:31.802058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:32.075754 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:32.075656    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:32.176627 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:32.176591    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:34.344437 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:34.344377    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:34.424224 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:34.424131    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:34.919726 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:34.919688    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:35.020882 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:35.020846    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:37.471686 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.471646    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kv8z4" event={"ID":"66a4327a-f8c0-4865-834d-f964c9b00a75","Type":"ContainerStarted","Data":"58e99010dc0f5c9c62c6e84386fee3fbb5adcf17f6c244d511f82ee29cd41fd7"}
Jul 21 12:52:37.639091 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.639053    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76596d6dc7-8gwst"]
Jul 21 12:52:37.658273 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.658238    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76596d6dc7-8gwst"]
Jul 21 12:52:37.658483 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.658390    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.667574 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.667517    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 21 12:52:37.788091 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.788049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-trusted-ca-bundle\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.788308 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.788102    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-oauth-config\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.788308 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.788188    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-serving-cert\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.788308 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.788219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-service-ca\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.788308 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.788256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-config\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.788308 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.788272    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mmf5\" (UniqueName: \"kubernetes.io/projected/f592e2fa-824f-4f2a-9560-94b95eef8a54-kube-api-access-6mmf5\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.788541 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.788390    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-oauth-serving-cert\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.889558 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.889473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-serving-cert\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.889558 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.889516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-service-ca\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.889558 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.889561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-config\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.889823 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.889581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mmf5\" (UniqueName: \"kubernetes.io/projected/f592e2fa-824f-4f2a-9560-94b95eef8a54-kube-api-access-6mmf5\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.889823 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.889624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-oauth-serving-cert\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.889823 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.889790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-trusted-ca-bundle\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.889969 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.889842    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-oauth-config\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.890532 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.890508    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-service-ca\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.890810 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.890691    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-trusted-ca-bundle\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.890938 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.890920    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-config\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.891849 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.891828    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-oauth-serving-cert\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.892490 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.892469    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-serving-cert\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.892647 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.892625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-oauth-config\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.900174 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.900144    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mmf5\" (UniqueName: \"kubernetes.io/projected/f592e2fa-824f-4f2a-9560-94b95eef8a54-kube-api-access-6mmf5\") pod \"console-76596d6dc7-8gwst\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") " pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:37.970540 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:37.970495    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:38.187527 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:38.187499    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76596d6dc7-8gwst"]
Jul 21 12:52:38.192376 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:52:38.192342    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf592e2fa_824f_4f2a_9560_94b95eef8a54.slice/crio-c48996c81e77852fe34cd87f16826423db5b6b5977e41378e8dabbd402980336 WatchSource:0}: Error finding container c48996c81e77852fe34cd87f16826423db5b6b5977e41378e8dabbd402980336: Status 404 returned error can't find the container with id c48996c81e77852fe34cd87f16826423db5b6b5977e41378e8dabbd402980336
Jul 21 12:52:38.479441 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:38.479361    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76596d6dc7-8gwst" event={"ID":"f592e2fa-824f-4f2a-9560-94b95eef8a54","Type":"ContainerStarted","Data":"cdbaba1b5842abc5e776a5d45bae607e98a335c44e8edfa62f3f75b68bd5b6ff"}
Jul 21 12:52:38.479441 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:38.479428    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76596d6dc7-8gwst" event={"ID":"f592e2fa-824f-4f2a-9560-94b95eef8a54","Type":"ContainerStarted","Data":"c48996c81e77852fe34cd87f16826423db5b6b5977e41378e8dabbd402980336"}
Jul 21 12:52:38.499236 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:38.499179    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76596d6dc7-8gwst" podStartSLOduration=1.49915791 podStartE2EDuration="1.49915791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:52:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:38.497578509 +0000 UTC m=+91.023081665" watchObservedRunningTime="2026-07-21 12:52:38.49915791 +0000 UTC m=+91.024661066"
Jul 21 12:52:39.156856 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:39.156819    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:39.218434 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:39.218402    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:39.424875 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:39.424759    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:39.485938 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:39.485881    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:41.162274 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:41.162239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:41.263264 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:41.263178    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:42.495762 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:42.495725    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kv8z4" event={"ID":"66a4327a-f8c0-4865-834d-f964c9b00a75","Type":"ContainerStarted","Data":"e10620865f5bfa24a94540d696d93a19637dfa33880deff5e5267b67cb50b857"}
Jul 21 12:52:47.516767 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:47.516721    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kv8z4" event={"ID":"66a4327a-f8c0-4865-834d-f964c9b00a75","Type":"ContainerStarted","Data":"30dc11667b80e29b1c7576b7a493ed897b5b4a105e76cee25082c2b0c1ca4cbb"}
Jul 21 12:52:47.540394 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:47.540345    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kv8z4" podStartSLOduration=17.413055604 podStartE2EDuration="33.540329622s" totalImagesPullingTime="16.127274018s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:52:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:52:47.538132855 +0000 UTC m=+100.063636010" watchObservedRunningTime="2026-07-21 12:52:47.540329622 +0000 UTC m=+100.065832777"
Jul 21 12:52:47.970982 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:47.970951    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:47.971199 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:47.971010    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:47.976355 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:47.976327    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:48.524925 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:48.524896    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:52:48.677862 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:48.677827    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:48.778525 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:48.778432    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:48.834318 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:48.834276    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:49.342993 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:49.342952    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:49.443885 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:49.443847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:52:52.391386 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:52.391350    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kpnng"
Jul 21 12:52:54.824038 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:54.823995    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78849cc87c-6dbtr" podUID="42b85580-cbe8-42da-a5b2-7d748e31cfc5" containerName="console" containerID="cri-o://3de5fe4249d3a978ea1bad2911c496ad5aec05490bafba2bc8fa720992e5b195" gracePeriod=15
Jul 21 12:52:55.106034 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.106001    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:55.130804 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.130779    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-serving-cert\") pod \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") "
Jul 21 12:52:55.130950 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.130820    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-oauth-serving-cert\") pod \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") "
Jul 21 12:52:55.130950 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.130854    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-service-ca\") pod \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") "
Jul 21 12:52:55.130950 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.130886    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mv4p\" (UniqueName: \"kubernetes.io/projected/42b85580-cbe8-42da-a5b2-7d748e31cfc5-kube-api-access-2mv4p\") pod \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") "
Jul 21 12:52:55.130950 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.130909    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-config\") pod \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") "
Jul 21 12:52:55.130950 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.130943    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-oauth-config\") pod \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\" (UID: \"42b85580-cbe8-42da-a5b2-7d748e31cfc5\") "
Jul 21 12:52:55.131358 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.131326    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "42b85580-cbe8-42da-a5b2-7d748e31cfc5" (UID: "42b85580-cbe8-42da-a5b2-7d748e31cfc5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:52:55.131358 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.131346    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-config" (OuterVolumeSpecName: "console-config") pod "42b85580-cbe8-42da-a5b2-7d748e31cfc5" (UID: "42b85580-cbe8-42da-a5b2-7d748e31cfc5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:52:55.131464 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.131366    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-service-ca" (OuterVolumeSpecName: "service-ca") pod "42b85580-cbe8-42da-a5b2-7d748e31cfc5" (UID: "42b85580-cbe8-42da-a5b2-7d748e31cfc5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:52:55.133530 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.133498    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42b85580-cbe8-42da-a5b2-7d748e31cfc5-kube-api-access-2mv4p" (OuterVolumeSpecName: "kube-api-access-2mv4p") pod "42b85580-cbe8-42da-a5b2-7d748e31cfc5" (UID: "42b85580-cbe8-42da-a5b2-7d748e31cfc5"). InnerVolumeSpecName "kube-api-access-2mv4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:52:55.133635 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.133621    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "42b85580-cbe8-42da-a5b2-7d748e31cfc5" (UID: "42b85580-cbe8-42da-a5b2-7d748e31cfc5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:52:55.133714 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.133697    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "42b85580-cbe8-42da-a5b2-7d748e31cfc5" (UID: "42b85580-cbe8-42da-a5b2-7d748e31cfc5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:52:55.231892 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.231855    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-serving-cert\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:55.231892 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.231884    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-oauth-serving-cert\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:55.231892 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.231893    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:55.231892 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.231901    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mv4p\" (UniqueName: \"kubernetes.io/projected/42b85580-cbe8-42da-a5b2-7d748e31cfc5-kube-api-access-2mv4p\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:55.232153 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.231910    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-config\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:55.232153 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.231919    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42b85580-cbe8-42da-a5b2-7d748e31cfc5-console-oauth-config\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:52:55.547617 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.547583    2563 generic.go:356] "Generic (PLEG): container finished" podID="42b85580-cbe8-42da-a5b2-7d748e31cfc5" containerID="3de5fe4249d3a978ea1bad2911c496ad5aec05490bafba2bc8fa720992e5b195" exitCode=0
Jul 21 12:52:55.547775 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.547627    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78849cc87c-6dbtr"
Jul 21 12:52:55.547775 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.547665    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78849cc87c-6dbtr" event={"ID":"42b85580-cbe8-42da-a5b2-7d748e31cfc5","Type":"ContainerDied","Data":"3de5fe4249d3a978ea1bad2911c496ad5aec05490bafba2bc8fa720992e5b195"}
Jul 21 12:52:55.547775 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.547715    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78849cc87c-6dbtr" event={"ID":"42b85580-cbe8-42da-a5b2-7d748e31cfc5","Type":"ContainerDied","Data":"ecf09a9aedbb63e8ed228c6f89d76128c1ea86e105fba4f0ecfda0a7a0ecff20"}
Jul 21 12:52:55.547775 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.547735    2563 scope.go:117] "RemoveContainer" containerID="3de5fe4249d3a978ea1bad2911c496ad5aec05490bafba2bc8fa720992e5b195"
Jul 21 12:52:55.556178 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.556162    2563 scope.go:117] "RemoveContainer" containerID="3de5fe4249d3a978ea1bad2911c496ad5aec05490bafba2bc8fa720992e5b195"
Jul 21 12:52:55.556463 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:52:55.556439    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3de5fe4249d3a978ea1bad2911c496ad5aec05490bafba2bc8fa720992e5b195\": container with ID starting with 3de5fe4249d3a978ea1bad2911c496ad5aec05490bafba2bc8fa720992e5b195 not found: ID does not exist" containerID="3de5fe4249d3a978ea1bad2911c496ad5aec05490bafba2bc8fa720992e5b195"
Jul 21 12:52:55.556519 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.556484    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de5fe4249d3a978ea1bad2911c496ad5aec05490bafba2bc8fa720992e5b195"} err="failed to get container status \"3de5fe4249d3a978ea1bad2911c496ad5aec05490bafba2bc8fa720992e5b195\": rpc error: code = NotFound desc = could not find container \"3de5fe4249d3a978ea1bad2911c496ad5aec05490bafba2bc8fa720992e5b195\": container with ID starting with 3de5fe4249d3a978ea1bad2911c496ad5aec05490bafba2bc8fa720992e5b195 not found: ID does not exist"
Jul 21 12:52:55.569845 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.569823    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78849cc87c-6dbtr"]
Jul 21 12:52:55.578549 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.578529    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78849cc87c-6dbtr"]
Jul 21 12:52:55.995270 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:52:55.995240    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42b85580-cbe8-42da-a5b2-7d748e31cfc5" path="/var/lib/kubelet/pods/42b85580-cbe8-42da-a5b2-7d748e31cfc5/volumes"
Jul 21 12:53:04.357762 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:04.357730    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:04.423644 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:04.423616    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:05.065964 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:05.065910    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:05.128173 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:05.128135    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:11.863173 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:11.863136    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:12.106522 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:12.106484    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:12.208019 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:12.207932    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:16.195472 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:16.195437    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:40.365118 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:40.365075    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:40.491892 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:40.491857    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:44.420140 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:44.420100    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:49.724403 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.724355    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77766884d5-27w7r"]
Jul 21 12:53:49.724777 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.724633    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42b85580-cbe8-42da-a5b2-7d748e31cfc5" containerName="console"
Jul 21 12:53:49.724777 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.724644    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="42b85580-cbe8-42da-a5b2-7d748e31cfc5" containerName="console"
Jul 21 12:53:49.724777 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.724685    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="42b85580-cbe8-42da-a5b2-7d748e31cfc5" containerName="console"
Jul 21 12:53:49.736560 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.736529    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.740446 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.740412    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77766884d5-27w7r"]
Jul 21 12:53:49.777623 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.777581    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-service-ca\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.777784 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.777630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-trusted-ca-bundle\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.777784 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.777663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-serving-cert\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.777784 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.777701    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-oauth-config\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.777987 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.777949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-config\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.778122 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.777997    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-oauth-serving-cert\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.778122 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.778095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm8w8\" (UniqueName: \"kubernetes.io/projected/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-kube-api-access-qm8w8\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.784246 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.784215    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:49.846328 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.846279    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:49.878495 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.878461    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm8w8\" (UniqueName: \"kubernetes.io/projected/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-kube-api-access-qm8w8\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.878629 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.878506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-service-ca\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.878629 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.878522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-trusted-ca-bundle\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.878727 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.878633    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-serving-cert\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.878727 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.878702    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-oauth-config\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.878837 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.878745    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-config\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.878837 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.878762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-oauth-serving-cert\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.879360 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.879330    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-service-ca\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.879491 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.879381    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-config\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.879491 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.879420    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-trusted-ca-bundle\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.879491 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.879444    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-oauth-serving-cert\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.881177 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.881148    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-serving-cert\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.881282 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.881229    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-oauth-config\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.890414 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.890391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm8w8\" (UniqueName: \"kubernetes.io/projected/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-kube-api-access-qm8w8\") pod \"console-77766884d5-27w7r\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") " pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:49.921339 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:49.921286    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:50.047551 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:50.047463    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:53:50.076246 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:50.076214    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:50.182596 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:50.182541    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77766884d5-27w7r"]
Jul 21 12:53:50.184733 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:53:50.184703    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod971ff5a0_d00a_4af4_a6aa_91b4cebc010f.slice/crio-ec6fcc7ea6ff88449ee29cd16b0f89785bbe5dd99ec3b992ec1b2f5117b88e25 WatchSource:0}: Error finding container ec6fcc7ea6ff88449ee29cd16b0f89785bbe5dd99ec3b992ec1b2f5117b88e25: Status 404 returned error can't find the container with id ec6fcc7ea6ff88449ee29cd16b0f89785bbe5dd99ec3b992ec1b2f5117b88e25
Jul 21 12:53:50.515557 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:50.515518    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:50.617157 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:50.617124    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:50.739819 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:50.739777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77766884d5-27w7r" event={"ID":"971ff5a0-d00a-4af4-a6aa-91b4cebc010f","Type":"ContainerStarted","Data":"97ff2af0a2e23d2c123d2c8b6637fde7a73645f84f44019a86cb4972a553f240"}
Jul 21 12:53:50.740176 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:50.739836    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77766884d5-27w7r" event={"ID":"971ff5a0-d00a-4af4-a6aa-91b4cebc010f","Type":"ContainerStarted","Data":"ec6fcc7ea6ff88449ee29cd16b0f89785bbe5dd99ec3b992ec1b2f5117b88e25"}
Jul 21 12:53:50.758178 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:50.758125    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77766884d5-27w7r" podStartSLOduration=1.7581109019999999 podStartE2EDuration="1.758110902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:53:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:53:50.756768207 +0000 UTC m=+163.282271363" watchObservedRunningTime="2026-07-21 12:53:50.758110902 +0000 UTC m=+163.283614097"
Jul 21 12:53:52.549801 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:52.549765    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:53:52.634972 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:53:52.634934    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:00.048145 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:00.048051    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:54:00.048145 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:00.048131    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:54:00.052943 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:00.052916    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:54:00.776753 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:00.776723    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:54:00.836910 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:00.836874    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76596d6dc7-8gwst"]
Jul 21 12:54:00.956061 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:00.956028    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:01.056204 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:01.056107    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:01.136247 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:01.136207    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:04.338262 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:04.338225    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:04.439238 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:04.439207    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:04.498305 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:04.498244    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:04.553377 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:04.553341    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:10.593457 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:10.593422    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:25.860960 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:25.860921    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76596d6dc7-8gwst" podUID="f592e2fa-824f-4f2a-9560-94b95eef8a54" containerName="console" containerID="cri-o://cdbaba1b5842abc5e776a5d45bae607e98a335c44e8edfa62f3f75b68bd5b6ff" gracePeriod=15
Jul 21 12:54:26.104191 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.104158    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:54:26.175638 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.175543    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-oauth-config\") pod \"f592e2fa-824f-4f2a-9560-94b95eef8a54\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") "
Jul 21 12:54:26.175638 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.175578    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-service-ca\") pod \"f592e2fa-824f-4f2a-9560-94b95eef8a54\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") "
Jul 21 12:54:26.175638 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.175599    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-config\") pod \"f592e2fa-824f-4f2a-9560-94b95eef8a54\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") "
Jul 21 12:54:26.175638 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.175633    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-trusted-ca-bundle\") pod \"f592e2fa-824f-4f2a-9560-94b95eef8a54\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") "
Jul 21 12:54:26.175956 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.175791    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-serving-cert\") pod \"f592e2fa-824f-4f2a-9560-94b95eef8a54\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") "
Jul 21 12:54:26.175956 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.175860    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mmf5\" (UniqueName: \"kubernetes.io/projected/f592e2fa-824f-4f2a-9560-94b95eef8a54-kube-api-access-6mmf5\") pod \"f592e2fa-824f-4f2a-9560-94b95eef8a54\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") "
Jul 21 12:54:26.175956 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.175924    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-oauth-serving-cert\") pod \"f592e2fa-824f-4f2a-9560-94b95eef8a54\" (UID: \"f592e2fa-824f-4f2a-9560-94b95eef8a54\") "
Jul 21 12:54:26.176108 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.176026    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-service-ca" (OuterVolumeSpecName: "service-ca") pod "f592e2fa-824f-4f2a-9560-94b95eef8a54" (UID: "f592e2fa-824f-4f2a-9560-94b95eef8a54"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:54:26.176108 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.176087    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-config" (OuterVolumeSpecName: "console-config") pod "f592e2fa-824f-4f2a-9560-94b95eef8a54" (UID: "f592e2fa-824f-4f2a-9560-94b95eef8a54"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:54:26.176108 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.176101    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f592e2fa-824f-4f2a-9560-94b95eef8a54" (UID: "f592e2fa-824f-4f2a-9560-94b95eef8a54"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:54:26.176388 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.176339    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:54:26.176388 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.176352    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f592e2fa-824f-4f2a-9560-94b95eef8a54" (UID: "f592e2fa-824f-4f2a-9560-94b95eef8a54"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:54:26.176388 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.176362    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-config\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:54:26.176388 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.176378    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-trusted-ca-bundle\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:54:26.177979 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.177954    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f592e2fa-824f-4f2a-9560-94b95eef8a54" (UID: "f592e2fa-824f-4f2a-9560-94b95eef8a54"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:54:26.177979 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.177965    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f592e2fa-824f-4f2a-9560-94b95eef8a54-kube-api-access-6mmf5" (OuterVolumeSpecName: "kube-api-access-6mmf5") pod "f592e2fa-824f-4f2a-9560-94b95eef8a54" (UID: "f592e2fa-824f-4f2a-9560-94b95eef8a54"). InnerVolumeSpecName "kube-api-access-6mmf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:54:26.178121 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.178029    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f592e2fa-824f-4f2a-9560-94b95eef8a54" (UID: "f592e2fa-824f-4f2a-9560-94b95eef8a54"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:54:26.277196 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.277160    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f592e2fa-824f-4f2a-9560-94b95eef8a54-oauth-serving-cert\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:54:26.277196 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.277187    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-oauth-config\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:54:26.277196 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.277196    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f592e2fa-824f-4f2a-9560-94b95eef8a54-console-serving-cert\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:54:26.277196 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.277204    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mmf5\" (UniqueName: \"kubernetes.io/projected/f592e2fa-824f-4f2a-9560-94b95eef8a54-kube-api-access-6mmf5\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:54:26.507185 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.507102    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:26.601386 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.601348    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:26.874556 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.874521    2563 generic.go:356] "Generic (PLEG): container finished" podID="f592e2fa-824f-4f2a-9560-94b95eef8a54" containerID="cdbaba1b5842abc5e776a5d45bae607e98a335c44e8edfa62f3f75b68bd5b6ff" exitCode=0
Jul 21 12:54:26.874942 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.874556    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76596d6dc7-8gwst" event={"ID":"f592e2fa-824f-4f2a-9560-94b95eef8a54","Type":"ContainerDied","Data":"cdbaba1b5842abc5e776a5d45bae607e98a335c44e8edfa62f3f75b68bd5b6ff"}
Jul 21 12:54:26.874942 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.874575    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76596d6dc7-8gwst"
Jul 21 12:54:26.874942 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.874614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76596d6dc7-8gwst" event={"ID":"f592e2fa-824f-4f2a-9560-94b95eef8a54","Type":"ContainerDied","Data":"c48996c81e77852fe34cd87f16826423db5b6b5977e41378e8dabbd402980336"}
Jul 21 12:54:26.874942 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.874639    2563 scope.go:117] "RemoveContainer" containerID="cdbaba1b5842abc5e776a5d45bae607e98a335c44e8edfa62f3f75b68bd5b6ff"
Jul 21 12:54:26.882946 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.882930    2563 scope.go:117] "RemoveContainer" containerID="cdbaba1b5842abc5e776a5d45bae607e98a335c44e8edfa62f3f75b68bd5b6ff"
Jul 21 12:54:26.883218 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:54:26.883201    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdbaba1b5842abc5e776a5d45bae607e98a335c44e8edfa62f3f75b68bd5b6ff\": container with ID starting with cdbaba1b5842abc5e776a5d45bae607e98a335c44e8edfa62f3f75b68bd5b6ff not found: ID does not exist" containerID="cdbaba1b5842abc5e776a5d45bae607e98a335c44e8edfa62f3f75b68bd5b6ff"
Jul 21 12:54:26.883257 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.883228    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdbaba1b5842abc5e776a5d45bae607e98a335c44e8edfa62f3f75b68bd5b6ff"} err="failed to get container status \"cdbaba1b5842abc5e776a5d45bae607e98a335c44e8edfa62f3f75b68bd5b6ff\": rpc error: code = NotFound desc = could not find container \"cdbaba1b5842abc5e776a5d45bae607e98a335c44e8edfa62f3f75b68bd5b6ff\": container with ID starting with cdbaba1b5842abc5e776a5d45bae607e98a335c44e8edfa62f3f75b68bd5b6ff not found: ID does not exist"
Jul 21 12:54:26.897227 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.897197    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76596d6dc7-8gwst"]
Jul 21 12:54:26.901438 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:26.901412    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76596d6dc7-8gwst"]
Jul 21 12:54:27.994680 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:27.994648    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f592e2fa-824f-4f2a-9560-94b95eef8a54" path="/var/lib/kubelet/pods/f592e2fa-824f-4f2a-9560-94b95eef8a54/volumes"
Jul 21 12:54:31.295008 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:31.294969    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:48.430545 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:48.430510    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:49.227524 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:49.227491    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:51.917184 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:51.917147    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:52.017825 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:52.017786    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:53.419507 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:53.419476    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:53.523377 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:53.523347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:53.755266 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:53.755176    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:53.867102 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:53.867068    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:54.346540 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.346506    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"]
Jul 21 12:54:54.346786 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.346772    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f592e2fa-824f-4f2a-9560-94b95eef8a54" containerName="console"
Jul 21 12:54:54.346881 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.346789    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f592e2fa-824f-4f2a-9560-94b95eef8a54" containerName="console"
Jul 21 12:54:54.346881 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.346845    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f592e2fa-824f-4f2a-9560-94b95eef8a54" containerName="console"
Jul 21 12:54:54.350064 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.350046    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"
Jul 21 12:54:54.352632 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.352608    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 21 12:54:54.352733 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.352654    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 21 12:54:54.352783 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.352740    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gwfgb\""
Jul 21 12:54:54.361519 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.361493    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"]
Jul 21 12:54:54.400829 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.400788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmqgx\" (UniqueName: \"kubernetes.io/projected/8c672098-87d3-447d-852b-9de5cf1eb0b8-kube-api-access-zmqgx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5\" (UID: \"8c672098-87d3-447d-852b-9de5cf1eb0b8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"
Jul 21 12:54:54.400829 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.400834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c672098-87d3-447d-852b-9de5cf1eb0b8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5\" (UID: \"8c672098-87d3-447d-852b-9de5cf1eb0b8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"
Jul 21 12:54:54.401040 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.400948    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c672098-87d3-447d-852b-9de5cf1eb0b8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5\" (UID: \"8c672098-87d3-447d-852b-9de5cf1eb0b8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"
Jul 21 12:54:54.502039 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.501996    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmqgx\" (UniqueName: \"kubernetes.io/projected/8c672098-87d3-447d-852b-9de5cf1eb0b8-kube-api-access-zmqgx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5\" (UID: \"8c672098-87d3-447d-852b-9de5cf1eb0b8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"
Jul 21 12:54:54.502039 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.502042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c672098-87d3-447d-852b-9de5cf1eb0b8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5\" (UID: \"8c672098-87d3-447d-852b-9de5cf1eb0b8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"
Jul 21 12:54:54.502489 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.502098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c672098-87d3-447d-852b-9de5cf1eb0b8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5\" (UID: \"8c672098-87d3-447d-852b-9de5cf1eb0b8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"
Jul 21 12:54:54.502565 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.502544    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c672098-87d3-447d-852b-9de5cf1eb0b8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5\" (UID: \"8c672098-87d3-447d-852b-9de5cf1eb0b8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"
Jul 21 12:54:54.502604 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.502559    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c672098-87d3-447d-852b-9de5cf1eb0b8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5\" (UID: \"8c672098-87d3-447d-852b-9de5cf1eb0b8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"
Jul 21 12:54:54.511854 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.511831    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmqgx\" (UniqueName: \"kubernetes.io/projected/8c672098-87d3-447d-852b-9de5cf1eb0b8-kube-api-access-zmqgx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5\" (UID: \"8c672098-87d3-447d-852b-9de5cf1eb0b8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"
Jul 21 12:54:54.659692 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.659654    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"
Jul 21 12:54:54.792867 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.792830    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"]
Jul 21 12:54:54.795315 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:54:54.795276    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c672098_87d3_447d_852b_9de5cf1eb0b8.slice/crio-2512112c9b4c59e778f65c7bf2e8c42ada6a2169e2d06f416eaf7c79141b3fdb WatchSource:0}: Error finding container 2512112c9b4c59e778f65c7bf2e8c42ada6a2169e2d06f416eaf7c79141b3fdb: Status 404 returned error can't find the container with id 2512112c9b4c59e778f65c7bf2e8c42ada6a2169e2d06f416eaf7c79141b3fdb
Jul 21 12:54:54.978595 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:54.978514    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5" event={"ID":"8c672098-87d3-447d-852b-9de5cf1eb0b8","Type":"ContainerStarted","Data":"2512112c9b4c59e778f65c7bf2e8c42ada6a2169e2d06f416eaf7c79141b3fdb"}
Jul 21 12:54:55.813194 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:55.813147    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:56.241165 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:56.241122    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:58.919493 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:58.919452    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:54:59.019991 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:54:59.019939    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:01.004079 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:01.004037    2563 generic.go:356] "Generic (PLEG): container finished" podID="8c672098-87d3-447d-852b-9de5cf1eb0b8" containerID="b0db7c569e6cd30dd1ae6e44583512b791fb4002b2969831081195195b28012f" exitCode=0
Jul 21 12:55:01.004491 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:01.004125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5" event={"ID":"8c672098-87d3-447d-852b-9de5cf1eb0b8","Type":"ContainerDied","Data":"b0db7c569e6cd30dd1ae6e44583512b791fb4002b2969831081195195b28012f"}
Jul 21 12:55:02.684993 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:02.684955    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:02.793280 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:02.793247    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:03.022577 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:03.022437    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:04.015612 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:04.015570    2563 generic.go:356] "Generic (PLEG): container finished" podID="8c672098-87d3-447d-852b-9de5cf1eb0b8" containerID="0718b69745f0334da5f50c12de3b0c880cf29c6b1f0a79c8d80eaa7a9359a901" exitCode=0
Jul 21 12:55:04.016070 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:04.015653    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5" event={"ID":"8c672098-87d3-447d-852b-9de5cf1eb0b8","Type":"ContainerDied","Data":"0718b69745f0334da5f50c12de3b0c880cf29c6b1f0a79c8d80eaa7a9359a901"}
Jul 21 12:55:11.047192 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:11.047156    2563 generic.go:356] "Generic (PLEG): container finished" podID="8c672098-87d3-447d-852b-9de5cf1eb0b8" containerID="3645d1b08c3194a342987ed58f93302143ea214bbbc4c9739951265dee640605" exitCode=0
Jul 21 12:55:11.047578 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:11.047240    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5" event={"ID":"8c672098-87d3-447d-852b-9de5cf1eb0b8","Type":"ContainerDied","Data":"3645d1b08c3194a342987ed58f93302143ea214bbbc4c9739951265dee640605"}
Jul 21 12:55:12.177864 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:12.177841    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"
Jul 21 12:55:12.255019 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:12.254991    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmqgx\" (UniqueName: \"kubernetes.io/projected/8c672098-87d3-447d-852b-9de5cf1eb0b8-kube-api-access-zmqgx\") pod \"8c672098-87d3-447d-852b-9de5cf1eb0b8\" (UID: \"8c672098-87d3-447d-852b-9de5cf1eb0b8\") "
Jul 21 12:55:12.255211 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:12.255066    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c672098-87d3-447d-852b-9de5cf1eb0b8-bundle\") pod \"8c672098-87d3-447d-852b-9de5cf1eb0b8\" (UID: \"8c672098-87d3-447d-852b-9de5cf1eb0b8\") "
Jul 21 12:55:12.255211 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:12.255166    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c672098-87d3-447d-852b-9de5cf1eb0b8-util\") pod \"8c672098-87d3-447d-852b-9de5cf1eb0b8\" (UID: \"8c672098-87d3-447d-852b-9de5cf1eb0b8\") "
Jul 21 12:55:12.255471 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:12.255452    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c672098-87d3-447d-852b-9de5cf1eb0b8-bundle" (OuterVolumeSpecName: "bundle") pod "8c672098-87d3-447d-852b-9de5cf1eb0b8" (UID: "8c672098-87d3-447d-852b-9de5cf1eb0b8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:55:12.257457 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:12.257425    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c672098-87d3-447d-852b-9de5cf1eb0b8-kube-api-access-zmqgx" (OuterVolumeSpecName: "kube-api-access-zmqgx") pod "8c672098-87d3-447d-852b-9de5cf1eb0b8" (UID: "8c672098-87d3-447d-852b-9de5cf1eb0b8"). InnerVolumeSpecName "kube-api-access-zmqgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:55:12.259416 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:12.259387    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c672098-87d3-447d-852b-9de5cf1eb0b8-util" (OuterVolumeSpecName: "util") pod "8c672098-87d3-447d-852b-9de5cf1eb0b8" (UID: "8c672098-87d3-447d-852b-9de5cf1eb0b8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:55:12.356165 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:12.356133    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c672098-87d3-447d-852b-9de5cf1eb0b8-util\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:55:12.356165 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:12.356166    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmqgx\" (UniqueName: \"kubernetes.io/projected/8c672098-87d3-447d-852b-9de5cf1eb0b8-kube-api-access-zmqgx\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:55:12.356390 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:12.356181    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c672098-87d3-447d-852b-9de5cf1eb0b8-bundle\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:55:13.055739 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:13.055706    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5" event={"ID":"8c672098-87d3-447d-852b-9de5cf1eb0b8","Type":"ContainerDied","Data":"2512112c9b4c59e778f65c7bf2e8c42ada6a2169e2d06f416eaf7c79141b3fdb"}
Jul 21 12:55:13.055901 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:13.055755    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2512112c9b4c59e778f65c7bf2e8c42ada6a2169e2d06f416eaf7c79141b3fdb"
Jul 21 12:55:13.055901 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:13.055755    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmskq5"
Jul 21 12:55:16.430457 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:16.430415    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:16.529733 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:16.529691    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:23.124187 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:23.124140    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:23.227065 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:23.227031    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:25.070477 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:25.070441    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:37.428754 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.428660    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:37.564961 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.564925    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"]
Jul 21 12:55:37.565235 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.565221    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c672098-87d3-447d-852b-9de5cf1eb0b8" containerName="extract"
Jul 21 12:55:37.565275 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.565239    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c672098-87d3-447d-852b-9de5cf1eb0b8" containerName="extract"
Jul 21 12:55:37.565275 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.565249    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c672098-87d3-447d-852b-9de5cf1eb0b8" containerName="util"
Jul 21 12:55:37.565275 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.565256    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c672098-87d3-447d-852b-9de5cf1eb0b8" containerName="util"
Jul 21 12:55:37.565275 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.565262    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c672098-87d3-447d-852b-9de5cf1eb0b8" containerName="pull"
Jul 21 12:55:37.565275 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.565267    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c672098-87d3-447d-852b-9de5cf1eb0b8" containerName="pull"
Jul 21 12:55:37.565446 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.565331    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c672098-87d3-447d-852b-9de5cf1eb0b8" containerName="extract"
Jul 21 12:55:37.573336 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.573319    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:37.576815 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.576788    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 21 12:55:37.577319 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.577280    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 21 12:55:37.577916 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.577902    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 21 12:55:37.578307 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.578279    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xrbw8\""
Jul 21 12:55:37.579797 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.579781    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:55:37.581024 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.581007    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 21 12:55:37.603522 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.603497    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"]
Jul 21 12:55:37.652833 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.652792    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:37.657043 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.657015    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/913e9809-59dd-42c1-94ce-15bbd88fdb57-metrics-cert\") pod \"lws-controller-manager-54f95887b7-ckjhk\" (UID: \"913e9809-59dd-42c1-94ce-15bbd88fdb57\") " pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:37.657167 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.657139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/913e9809-59dd-42c1-94ce-15bbd88fdb57-cert\") pod \"lws-controller-manager-54f95887b7-ckjhk\" (UID: \"913e9809-59dd-42c1-94ce-15bbd88fdb57\") " pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:37.657204 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.657163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/913e9809-59dd-42c1-94ce-15bbd88fdb57-manager-config\") pod \"lws-controller-manager-54f95887b7-ckjhk\" (UID: \"913e9809-59dd-42c1-94ce-15bbd88fdb57\") " pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:37.657204 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.657193    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbvkd\" (UniqueName: \"kubernetes.io/projected/913e9809-59dd-42c1-94ce-15bbd88fdb57-kube-api-access-dbvkd\") pod \"lws-controller-manager-54f95887b7-ckjhk\" (UID: \"913e9809-59dd-42c1-94ce-15bbd88fdb57\") " pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:37.758441 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.758359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/913e9809-59dd-42c1-94ce-15bbd88fdb57-cert\") pod \"lws-controller-manager-54f95887b7-ckjhk\" (UID: \"913e9809-59dd-42c1-94ce-15bbd88fdb57\") " pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:37.758441 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.758405    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/913e9809-59dd-42c1-94ce-15bbd88fdb57-manager-config\") pod \"lws-controller-manager-54f95887b7-ckjhk\" (UID: \"913e9809-59dd-42c1-94ce-15bbd88fdb57\") " pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:37.758557 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.758441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbvkd\" (UniqueName: \"kubernetes.io/projected/913e9809-59dd-42c1-94ce-15bbd88fdb57-kube-api-access-dbvkd\") pod \"lws-controller-manager-54f95887b7-ckjhk\" (UID: \"913e9809-59dd-42c1-94ce-15bbd88fdb57\") " pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:37.758557 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.758471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/913e9809-59dd-42c1-94ce-15bbd88fdb57-metrics-cert\") pod \"lws-controller-manager-54f95887b7-ckjhk\" (UID: \"913e9809-59dd-42c1-94ce-15bbd88fdb57\") " pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:37.759156 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.759096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/913e9809-59dd-42c1-94ce-15bbd88fdb57-manager-config\") pod \"lws-controller-manager-54f95887b7-ckjhk\" (UID: \"913e9809-59dd-42c1-94ce-15bbd88fdb57\") " pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:37.761000 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.760977    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/913e9809-59dd-42c1-94ce-15bbd88fdb57-cert\") pod \"lws-controller-manager-54f95887b7-ckjhk\" (UID: \"913e9809-59dd-42c1-94ce-15bbd88fdb57\") " pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:37.761155 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.761142    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/913e9809-59dd-42c1-94ce-15bbd88fdb57-metrics-cert\") pod \"lws-controller-manager-54f95887b7-ckjhk\" (UID: \"913e9809-59dd-42c1-94ce-15bbd88fdb57\") " pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:37.773152 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.773123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbvkd\" (UniqueName: \"kubernetes.io/projected/913e9809-59dd-42c1-94ce-15bbd88fdb57-kube-api-access-dbvkd\") pod \"lws-controller-manager-54f95887b7-ckjhk\" (UID: \"913e9809-59dd-42c1-94ce-15bbd88fdb57\") " pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:37.882610 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:37.882548    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:38.019245 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:38.019214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"]
Jul 21 12:55:38.021821 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:55:38.021796    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod913e9809_59dd_42c1_94ce_15bbd88fdb57.slice/crio-bc9ed24df82f48e9f3c679dc7facd996d9e17f6fa107590279cdcb85ee31c019 WatchSource:0}: Error finding container bc9ed24df82f48e9f3c679dc7facd996d9e17f6fa107590279cdcb85ee31c019: Status 404 returned error can't find the container with id bc9ed24df82f48e9f3c679dc7facd996d9e17f6fa107590279cdcb85ee31c019
Jul 21 12:55:38.143984 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:38.143951    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk" event={"ID":"913e9809-59dd-42c1-94ce-15bbd88fdb57","Type":"ContainerStarted","Data":"bc9ed24df82f48e9f3c679dc7facd996d9e17f6fa107590279cdcb85ee31c019"}
Jul 21 12:55:40.152558 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:40.152523    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk" event={"ID":"913e9809-59dd-42c1-94ce-15bbd88fdb57","Type":"ContainerStarted","Data":"4c20457c65557151df84e5a4efe83d415604c1c1e7fbf90267e4d374ae731ee6"}
Jul 21 12:55:40.152934 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:40.152699    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:40.187769 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:40.187723    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk" podStartSLOduration=1.758777663 podStartE2EDuration="3.187707913s" totalImagesPullingTime="1.42893025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:55:40.185860141 +0000 UTC m=+272.711363307" watchObservedRunningTime="2026-07-21 12:55:40.187707913 +0000 UTC m=+272.713211068"
Jul 21 12:55:41.150003 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:41.149962    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:41.313782 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:41.313750    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:41.760631 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:41.760598    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:42.014253 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.014169    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:42.121469 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.121435    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:42.200465 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.200433    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"]
Jul 21 12:55:42.216211 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.216180    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.219037 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.219016    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 21 12:55:42.219717 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.219697    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-w8lq5\""
Jul 21 12:55:42.219717 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.219713    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 21 12:55:42.219717 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.219696    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 21 12:55:42.219958 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.219697    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 21 12:55:42.227767 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.227743    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"]
Jul 21 12:55:42.373057 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.373024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:42.393229 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.393205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/825324e5-8fed-402e-9006-0ff3ddd0ec07-tmp\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.393408 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.393247    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/825324e5-8fed-402e-9006-0ff3ddd0ec07-manifests\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.393408 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.393274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/825324e5-8fed-402e-9006-0ff3ddd0ec07-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.393408 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.393365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/825324e5-8fed-402e-9006-0ff3ddd0ec07-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.393509 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.393470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26nvx\" (UniqueName: \"kubernetes.io/projected/825324e5-8fed-402e-9006-0ff3ddd0ec07-kube-api-access-26nvx\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.474206 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.474171    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:42.494341 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.494281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26nvx\" (UniqueName: \"kubernetes.io/projected/825324e5-8fed-402e-9006-0ff3ddd0ec07-kube-api-access-26nvx\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.494507 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.494349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/825324e5-8fed-402e-9006-0ff3ddd0ec07-tmp\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.494507 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.494384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/825324e5-8fed-402e-9006-0ff3ddd0ec07-manifests\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.494507 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.494405    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/825324e5-8fed-402e-9006-0ff3ddd0ec07-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.494507 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.494444    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/825324e5-8fed-402e-9006-0ff3ddd0ec07-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.494887 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.494857    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/825324e5-8fed-402e-9006-0ff3ddd0ec07-manifests\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.496758 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.496729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/825324e5-8fed-402e-9006-0ff3ddd0ec07-tmp\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.496987 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.496968    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/825324e5-8fed-402e-9006-0ff3ddd0ec07-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.497057 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.496973    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/825324e5-8fed-402e-9006-0ff3ddd0ec07-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.504708 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.504684    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26nvx\" (UniqueName: \"kubernetes.io/projected/825324e5-8fed-402e-9006-0ff3ddd0ec07-kube-api-access-26nvx\") pod \"opendatahub-operator-controller-manager-59f884b5-jjwjp\" (UID: \"825324e5-8fed-402e-9006-0ff3ddd0ec07\") " pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.527702 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.527666    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:42.573495 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.573459    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:42.663174 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.663139    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:42.688930 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.688908    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"]
Jul 21 12:55:42.691128 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:55:42.691103    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod825324e5_8fed_402e_9006_0ff3ddd0ec07.slice/crio-bbbdd61719dd7c7a5e70b10bb9325d957b369c736c6658e3f53d5935ae6a515e WatchSource:0}: Error finding container bbbdd61719dd7c7a5e70b10bb9325d957b369c736c6658e3f53d5935ae6a515e: Status 404 returned error can't find the container with id bbbdd61719dd7c7a5e70b10bb9325d957b369c736c6658e3f53d5935ae6a515e
Jul 21 12:55:42.727519 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.727483    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:42.830923 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:42.830891    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:55:43.164589 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:43.164556    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp" event={"ID":"825324e5-8fed-402e-9006-0ff3ddd0ec07","Type":"ContainerStarted","Data":"bbbdd61719dd7c7a5e70b10bb9325d957b369c736c6658e3f53d5935ae6a515e"}
Jul 21 12:55:46.179341 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:46.179280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp" event={"ID":"825324e5-8fed-402e-9006-0ff3ddd0ec07","Type":"ContainerStarted","Data":"c8d4e3e8c122830fa8a04db2891d8a251a18648980147c91b888453de35ab635"}
Jul 21 12:55:47.189430 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:47.189390    2563 generic.go:356] "Generic (PLEG): container finished" podID="825324e5-8fed-402e-9006-0ff3ddd0ec07" containerID="c8d4e3e8c122830fa8a04db2891d8a251a18648980147c91b888453de35ab635" exitCode=0
Jul 21 12:55:47.189899 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:47.189444    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp" event={"ID":"825324e5-8fed-402e-9006-0ff3ddd0ec07","Type":"ContainerDied","Data":"c8d4e3e8c122830fa8a04db2891d8a251a18648980147c91b888453de35ab635"}
Jul 21 12:55:48.196721 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:48.196679    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp" event={"ID":"825324e5-8fed-402e-9006-0ff3ddd0ec07","Type":"ContainerStarted","Data":"db4307429856f2ba0f5a6eb19a7498d504015a84d353ce7ae9f85d3c326eca5b"}
Jul 21 12:55:48.197176 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:48.196815    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:55:48.239701 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:48.239647    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp" podStartSLOduration=1.889758129 podStartE2EDuration="6.23962892s" totalImagesPullingTime="3.278733893s" totalInitContainerRuntime="1.071136898s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:55:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:55:48.237417504 +0000 UTC m=+280.762920669" watchObservedRunningTime="2026-07-21 12:55:48.23962892 +0000 UTC m=+280.765132111"
Jul 21 12:55:51.158427 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:51.158399    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-54f95887b7-ckjhk"
Jul 21 12:55:59.202828 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:55:59.202792    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59f884b5-jjwjp"
Jul 21 12:56:01.672802 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:01.672751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:03.366951 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:03.366897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:03.925475 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:03.925435    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:04.118603 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:04.118564    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:04.409590 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:04.409555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:04.520205 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:04.520173    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:06.004210 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:06.004150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:07.913066 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:07.913040    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 12:56:07.914847 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:07.914824    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 12:56:07.916900 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:07.916880    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 21 12:56:09.440635 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.440584    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:09.539007 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.538976    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:09.672625 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.672591    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"]
Jul 21 12:56:09.676743 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.676721    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.679635 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.679615    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zcjbf\""
Jul 21 12:56:09.680161 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.680145    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 21 12:56:09.690736 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.690675    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"]
Jul 21 12:56:09.700407 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.700377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.700560 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.700420    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.700560 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.700448    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk72g\" (UniqueName: \"kubernetes.io/projected/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-kube-api-access-dk72g\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.700560 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.700524    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.700725 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.700568    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.700725 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.700597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.700725 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.700674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.700849 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.700729    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.700849 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.700767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.742191 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.742155    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:09.801649 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.801608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.801839 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.801681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.801839 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.801717    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.801839 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.801748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.801839 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.801786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.801839 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.801813    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.802069 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.801861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.802069 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.801888    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.802069 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.801914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk72g\" (UniqueName: \"kubernetes.io/projected/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-kube-api-access-dk72g\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.802069 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.802057    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.802432 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.802405    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.802579 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.802553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.802857 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.802799    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.802989 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.802963    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.804956 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.804910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.805285 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.805271    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.815680 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.815655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.830240 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.830215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk72g\" (UniqueName: \"kubernetes.io/projected/e73d90e5-0737-4c57-9ecd-6fb323e6c11f-kube-api-access-dk72g\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678jb2hm\" (UID: \"e73d90e5-0737-4c57-9ecd-6fb323e6c11f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:09.862073 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.861981    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:09.987773 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:09.987695    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:10.006701 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.006669    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:10.166106 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.166077    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"]
Jul 21 12:56:10.167892 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:56:10.167869    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode73d90e5_0737_4c57_9ecd_6fb323e6c11f.slice/crio-54c36c2216318dde0de0e8ceb9fc67b413f86c211f97175f02b1edc425e67ef9 WatchSource:0}: Error finding container 54c36c2216318dde0de0e8ceb9fc67b413f86c211f97175f02b1edc425e67ef9: Status 404 returned error can't find the container with id 54c36c2216318dde0de0e8ceb9fc67b413f86c211f97175f02b1edc425e67ef9
Jul 21 12:56:10.169917 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.169900    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:56:10.290668 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.290577    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm" event={"ID":"e73d90e5-0737-4c57-9ecd-6fb323e6c11f","Type":"ContainerStarted","Data":"54c36c2216318dde0de0e8ceb9fc67b413f86c211f97175f02b1edc425e67ef9"}
Jul 21 12:56:10.379869 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.379835    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"]
Jul 21 12:56:10.385267 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.385244    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.389178 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.389154    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 21 12:56:10.389419 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.389404    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-mqnjn\""
Jul 21 12:56:10.389718 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.389701    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 21 12:56:10.390241 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.390227    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 21 12:56:10.403236 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.403213    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"]
Jul 21 12:56:10.405928 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.405906    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/15780951-a3f8-45b5-ae67-1a8e47d0462a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.406051 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.405942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/15780951-a3f8-45b5-ae67-1a8e47d0462a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.406051 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.405969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/15780951-a3f8-45b5-ae67-1a8e47d0462a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.406051 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.406005    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9zzj\" (UniqueName: \"kubernetes.io/projected/15780951-a3f8-45b5-ae67-1a8e47d0462a-kube-api-access-m9zzj\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.406051 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.406045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/15780951-a3f8-45b5-ae67-1a8e47d0462a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.406239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.406140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15780951-a3f8-45b5-ae67-1a8e47d0462a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.406239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.406166    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/15780951-a3f8-45b5-ae67-1a8e47d0462a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.507215 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.507176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/15780951-a3f8-45b5-ae67-1a8e47d0462a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.507215 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.507217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/15780951-a3f8-45b5-ae67-1a8e47d0462a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.507866 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.507246    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9zzj\" (UniqueName: \"kubernetes.io/projected/15780951-a3f8-45b5-ae67-1a8e47d0462a-kube-api-access-m9zzj\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.507866 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.507274    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/15780951-a3f8-45b5-ae67-1a8e47d0462a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.507866 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.507442    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15780951-a3f8-45b5-ae67-1a8e47d0462a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.507866 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.507468    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/15780951-a3f8-45b5-ae67-1a8e47d0462a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.507866 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.507688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/15780951-a3f8-45b5-ae67-1a8e47d0462a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.508859 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.508833    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/15780951-a3f8-45b5-ae67-1a8e47d0462a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.509979 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.509954    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/15780951-a3f8-45b5-ae67-1a8e47d0462a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.510103 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.510089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/15780951-a3f8-45b5-ae67-1a8e47d0462a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.510269 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.510252    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/15780951-a3f8-45b5-ae67-1a8e47d0462a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.510969 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.510913    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/15780951-a3f8-45b5-ae67-1a8e47d0462a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.531156 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.531123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15780951-a3f8-45b5-ae67-1a8e47d0462a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.531462 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.531446    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9zzj\" (UniqueName: \"kubernetes.io/projected/15780951-a3f8-45b5-ae67-1a8e47d0462a-kube-api-access-m9zzj\") pod \"istiod-openshift-gateway-94bb8fbfd-s5xhb\" (UID: \"15780951-a3f8-45b5-ae67-1a8e47d0462a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.694285 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.694249    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:10.882794 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:10.882758    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"]
Jul 21 12:56:10.885706 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:56:10.885674    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15780951_a3f8_45b5_ae67_1a8e47d0462a.slice/crio-546ae368f02a872a83e2d7cbd2ae3e1a0064558cf1c4b87d74872e56ac80eea1 WatchSource:0}: Error finding container 546ae368f02a872a83e2d7cbd2ae3e1a0064558cf1c4b87d74872e56ac80eea1: Status 404 returned error can't find the container with id 546ae368f02a872a83e2d7cbd2ae3e1a0064558cf1c4b87d74872e56ac80eea1
Jul 21 12:56:11.297239 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:11.297204    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb" event={"ID":"15780951-a3f8-45b5-ae67-1a8e47d0462a","Type":"ContainerStarted","Data":"546ae368f02a872a83e2d7cbd2ae3e1a0064558cf1c4b87d74872e56ac80eea1"}
Jul 21 12:56:13.468080 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:13.468047    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:13.644036 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:13.643964    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:13.767419 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:13.767339    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:13.806193 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:13.806157    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:56:13.806323 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:13.806228    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:56:13.806323 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:13.806257    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:56:13.806657 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:13.806633    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:56:13.806705 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:13.806685    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:56:13.990352 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:13.990315    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:14.311188 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:14.311148    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm" event={"ID":"e73d90e5-0737-4c57-9ecd-6fb323e6c11f","Type":"ContainerStarted","Data":"63e826febccc7269dbf9bf07108197e00c8c259c40dafac1bf61bd06db642e14"}
Jul 21 12:56:14.312827 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:14.312802    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb" event={"ID":"15780951-a3f8-45b5-ae67-1a8e47d0462a","Type":"ContainerStarted","Data":"37c70afde66c725c56771f0a565ee81bc88765af0c43fb5734e5618b07a4595a"}
Jul 21 12:56:14.312969 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:14.312951    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:14.314745 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:14.314721    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-s5xhb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 12:56:14.314835 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:14.314777    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb" podUID="15780951-a3f8-45b5-ae67-1a8e47d0462a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 12:56:14.349590 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:14.349540    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm" podStartSLOduration=1.713692806 podStartE2EDuration="5.349524184s" totalImagesPullingTime="3.635831378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:56:14.347072048 +0000 UTC m=+306.872575205" watchObservedRunningTime="2026-07-21 12:56:14.349524184 +0000 UTC m=+306.875027341"
Jul 21 12:56:14.384514 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:14.384450    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb" podStartSLOduration=1.466106898 podStartE2EDuration="4.384432654s" totalImagesPullingTime="2.918325756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:56:14.38350209 +0000 UTC m=+306.909005245" watchObservedRunningTime="2026-07-21 12:56:14.384432654 +0000 UTC m=+306.909935811"
Jul 21 12:56:14.899582 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:14.899535    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:14.988680 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:14.988644    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:14.993322 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:14.993280    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:15.046629 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:15.046600    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:56:15.317274 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:15.317181    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:56:15.317670 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:15.317649    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-s5xhb"
Jul 21 12:56:15.318214 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:56:15.318198    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678jb2hm"
Jul 21 12:57:03.148280 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:03.148197    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:03.225004 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:03.224964    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:03.378541 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:03.378510    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:04.069328 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:04.069278    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:04.264973 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:04.264941    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:10.214998 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:10.214957    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:10.315643 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:10.315600    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:15.339479 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:15.339434    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:15.401635 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:15.401603    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:17.090150 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:17.090113    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:17.171155 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:17.171124    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:33.219193 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:33.219163    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:36.582371 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:36.582334    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:36.691951 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:36.691915    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:38.115007 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:38.114976    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:38.262057 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:38.262014    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:41.214966 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:41.214933    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:41.580019 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:41.579925    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:43.067638 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:43.067601    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:44.577373 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:44.577341    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:45.456040 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:45.456008    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:45.555112 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:45.555082    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:46.451255 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:46.451229    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:46.551715 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:46.551679    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:47.504768 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:47.504730    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:48.800768 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:48.800724    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:48.902615 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:48.902581    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:49.255414 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:49.255383    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77766884d5-27w7r"]
Jul 21 12:57:49.411955 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:49.411923    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:49.513668 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:49.513582    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:49.613443 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:49.613392    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:50.442342 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:50.442308    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:50.654709 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:50.654674    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:52.188827 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:52.188784    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:57:52.291237 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:57:52.291204    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:00.845382 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:00.845342    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:00.974541 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:00.974504    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:01.069543 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:01.069509    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:01.169107 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:01.169068    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:03.268607 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:03.268563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:03.371660 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:03.371628    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:07.575430 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:07.575399    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:07.680751 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:07.680720    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:09.813052 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:09.813015    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:09.915773 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:09.915737    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:14.283534 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.283472    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77766884d5-27w7r" podUID="971ff5a0-d00a-4af4-a6aa-91b4cebc010f" containerName="console" containerID="cri-o://97ff2af0a2e23d2c123d2c8b6637fde7a73645f84f44019a86cb4972a553f240" gracePeriod=15
Jul 21 12:58:14.525584 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.525559    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:58:14.538786 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.538719    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-oauth-config\") pod \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") "
Jul 21 12:58:14.538786 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.538761    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-oauth-serving-cert\") pod \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") "
Jul 21 12:58:14.538967 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.538800    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm8w8\" (UniqueName: \"kubernetes.io/projected/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-kube-api-access-qm8w8\") pod \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") "
Jul 21 12:58:14.538967 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.538817    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-config\") pod \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") "
Jul 21 12:58:14.539166 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.539140    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "971ff5a0-d00a-4af4-a6aa-91b4cebc010f" (UID: "971ff5a0-d00a-4af4-a6aa-91b4cebc010f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:58:14.539268 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.539240    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-config" (OuterVolumeSpecName: "console-config") pod "971ff5a0-d00a-4af4-a6aa-91b4cebc010f" (UID: "971ff5a0-d00a-4af4-a6aa-91b4cebc010f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:58:14.540916 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.540895    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "971ff5a0-d00a-4af4-a6aa-91b4cebc010f" (UID: "971ff5a0-d00a-4af4-a6aa-91b4cebc010f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:58:14.541006 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.540932    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-kube-api-access-qm8w8" (OuterVolumeSpecName: "kube-api-access-qm8w8") pod "971ff5a0-d00a-4af4-a6aa-91b4cebc010f" (UID: "971ff5a0-d00a-4af4-a6aa-91b4cebc010f"). InnerVolumeSpecName "kube-api-access-qm8w8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:58:14.639466 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.639427    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-service-ca\") pod \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") "
Jul 21 12:58:14.639466 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.639466    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-serving-cert\") pod \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") "
Jul 21 12:58:14.639720 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.639487    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-trusted-ca-bundle\") pod \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\" (UID: \"971ff5a0-d00a-4af4-a6aa-91b4cebc010f\") "
Jul 21 12:58:14.639777 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.639722    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-oauth-config\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:58:14.639777 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.639740    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-oauth-serving-cert\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:58:14.639777 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.639755    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm8w8\" (UniqueName: \"kubernetes.io/projected/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-kube-api-access-qm8w8\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:58:14.639777 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.639770    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-config\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:58:14.639930 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.639788    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-service-ca" (OuterVolumeSpecName: "service-ca") pod "971ff5a0-d00a-4af4-a6aa-91b4cebc010f" (UID: "971ff5a0-d00a-4af4-a6aa-91b4cebc010f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:58:14.639930 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.639856    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "971ff5a0-d00a-4af4-a6aa-91b4cebc010f" (UID: "971ff5a0-d00a-4af4-a6aa-91b4cebc010f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:58:14.641776 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.641754    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "971ff5a0-d00a-4af4-a6aa-91b4cebc010f" (UID: "971ff5a0-d00a-4af4-a6aa-91b4cebc010f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:58:14.740243 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.740205    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:58:14.740243 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.740234    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-console-serving-cert\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:58:14.740243 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.740243    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/971ff5a0-d00a-4af4-a6aa-91b4cebc010f-trusted-ca-bundle\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 12:58:14.802693 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.802609    2563 generic.go:356] "Generic (PLEG): container finished" podID="971ff5a0-d00a-4af4-a6aa-91b4cebc010f" containerID="97ff2af0a2e23d2c123d2c8b6637fde7a73645f84f44019a86cb4972a553f240" exitCode=0
Jul 21 12:58:14.802693 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.802658    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77766884d5-27w7r"
Jul 21 12:58:14.802863 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.802699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77766884d5-27w7r" event={"ID":"971ff5a0-d00a-4af4-a6aa-91b4cebc010f","Type":"ContainerDied","Data":"97ff2af0a2e23d2c123d2c8b6637fde7a73645f84f44019a86cb4972a553f240"}
Jul 21 12:58:14.802863 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.802752    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77766884d5-27w7r" event={"ID":"971ff5a0-d00a-4af4-a6aa-91b4cebc010f","Type":"ContainerDied","Data":"ec6fcc7ea6ff88449ee29cd16b0f89785bbe5dd99ec3b992ec1b2f5117b88e25"}
Jul 21 12:58:14.802863 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.802778    2563 scope.go:117] "RemoveContainer" containerID="97ff2af0a2e23d2c123d2c8b6637fde7a73645f84f44019a86cb4972a553f240"
Jul 21 12:58:14.811981 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.811966    2563 scope.go:117] "RemoveContainer" containerID="97ff2af0a2e23d2c123d2c8b6637fde7a73645f84f44019a86cb4972a553f240"
Jul 21 12:58:14.812223 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:58:14.812207    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97ff2af0a2e23d2c123d2c8b6637fde7a73645f84f44019a86cb4972a553f240\": container with ID starting with 97ff2af0a2e23d2c123d2c8b6637fde7a73645f84f44019a86cb4972a553f240 not found: ID does not exist" containerID="97ff2af0a2e23d2c123d2c8b6637fde7a73645f84f44019a86cb4972a553f240"
Jul 21 12:58:14.812270 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.812230    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97ff2af0a2e23d2c123d2c8b6637fde7a73645f84f44019a86cb4972a553f240"} err="failed to get container status \"97ff2af0a2e23d2c123d2c8b6637fde7a73645f84f44019a86cb4972a553f240\": rpc error: code = NotFound desc = could not find container \"97ff2af0a2e23d2c123d2c8b6637fde7a73645f84f44019a86cb4972a553f240\": container with ID starting with 97ff2af0a2e23d2c123d2c8b6637fde7a73645f84f44019a86cb4972a553f240 not found: ID does not exist"
Jul 21 12:58:14.825674 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.825643    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77766884d5-27w7r"]
Jul 21 12:58:14.832979 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:14.832952    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77766884d5-27w7r"]
Jul 21 12:58:15.995152 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:15.995119    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="971ff5a0-d00a-4af4-a6aa-91b4cebc010f" path="/var/lib/kubelet/pods/971ff5a0-d00a-4af4-a6aa-91b4cebc010f/volumes"
Jul 21 12:58:18.553204 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:18.553165    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:18.651282 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:18.651251    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:20.994399 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:20.994362    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:21.056756 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:21.056720    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:25.482154 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:25.482094    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:25.582118 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:25.582076    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:28.146385 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:28.146274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:28.208674 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:28.208641    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:30.771796 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.771734    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"]
Jul 21 12:58:30.772215 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.772198    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="971ff5a0-d00a-4af4-a6aa-91b4cebc010f" containerName="console"
Jul 21 12:58:30.772262 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.772217    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="971ff5a0-d00a-4af4-a6aa-91b4cebc010f" containerName="console"
Jul 21 12:58:30.772325 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.772313    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="971ff5a0-d00a-4af4-a6aa-91b4cebc010f" containerName="console"
Jul 21 12:58:30.775346 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.775323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.778079 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.778059    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2d7vr\""
Jul 21 12:58:30.790521 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.790491    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"]
Jul 21 12:58:30.810423 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.810380    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:30.871353 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.871313    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.871534 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.871372    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.871534 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.871475    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.871534 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.871521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.871706 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.871579    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsk7k\" (UniqueName: \"kubernetes.io/projected/ff529716-9b98-4184-b2ef-9ae9e1b3e428-kube-api-access-tsk7k\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.871706 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.871617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.871706 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.871647    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.871706 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.871676    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.871867 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.871705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.972736 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.972694    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsk7k\" (UniqueName: \"kubernetes.io/projected/ff529716-9b98-4184-b2ef-9ae9e1b3e428-kube-api-access-tsk7k\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.972736 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.972739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.973025 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.972758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.973025 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.972777    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.973025 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.972795    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.973025 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.972864    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.973025 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.972911    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.973025 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.972945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.973025 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.972973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.973403 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.973181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.973403 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.973256    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.973496 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.973464    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.973648 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.973621    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.973804 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.973785    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.975655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.975624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.975785 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.975763    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.981614 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.981585    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsk7k\" (UniqueName: \"kubernetes.io/projected/ff529716-9b98-4184-b2ef-9ae9e1b3e428-kube-api-access-tsk7k\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:30.981924 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:30.981895    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff529716-9b98-4184-b2ef-9ae9e1b3e428-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-xxfdb\" (UID: \"ff529716-9b98-4184-b2ef-9ae9e1b3e428\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:31.088929 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:31.088822    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:31.216528 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:31.216496    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:31.234342 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:31.234310    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"]
Jul 21 12:58:31.236967 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:58:31.236943    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff529716_9b98_4184_b2ef_9ae9e1b3e428.slice/crio-1537302f259fbd0fdd4667f6e88b3b8a0a62844b58526c702eb512d0d7d026fd WatchSource:0}: Error finding container 1537302f259fbd0fdd4667f6e88b3b8a0a62844b58526c702eb512d0d7d026fd: Status 404 returned error can't find the container with id 1537302f259fbd0fdd4667f6e88b3b8a0a62844b58526c702eb512d0d7d026fd
Jul 21 12:58:31.239123 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:31.239084    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:58:31.239201 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:31.239154    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:58:31.239201 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:31.239186    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:58:31.876910 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:31.876871    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb" event={"ID":"ff529716-9b98-4184-b2ef-9ae9e1b3e428","Type":"ContainerStarted","Data":"45116e920c61df7f0c9a4401a8d82d064854341aab57dddd5ef574a4aa037660"}
Jul 21 12:58:31.877312 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:31.876934    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb" event={"ID":"ff529716-9b98-4184-b2ef-9ae9e1b3e428","Type":"ContainerStarted","Data":"1537302f259fbd0fdd4667f6e88b3b8a0a62844b58526c702eb512d0d7d026fd"}
Jul 21 12:58:31.900723 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:31.900669    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb" podStartSLOduration=1.9006518049999999 podStartE2EDuration="1.900651805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:58:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:58:31.899237448 +0000 UTC m=+444.424740614" watchObservedRunningTime="2026-07-21 12:58:31.900651805 +0000 UTC m=+444.426154959"
Jul 21 12:58:32.089397 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:32.089342    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:32.094453 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:32.094428    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:32.881351 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:32.881320    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:32.882237 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:32.882214    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 12:58:32.982159 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:32.982120    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:33.096439 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:33.096409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:33.162360 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:33.162250    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:33.893903 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:33.893869    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:33.957834 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:33.957798    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:37.358341 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:37.358272    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:37.457839 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:37.457808    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:39.815356 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:39.815320    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:39.878873 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:39.878831    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:54.134645 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:54.134611    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:54.229449 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:54.229409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:54.329634 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:54.329600    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:58:54.603598 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:58:54.603568    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:07.008433 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:07.008398    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:07.172517 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:07.172488    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:12.047330 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:12.047274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:12.148441 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:12.148407    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:14.426334 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.426280    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:14.548982 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.548936    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:14.644653 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.644620    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-688696d74-9787c"]
Jul 21 12:59:14.648994 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.648961    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:14.649479 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.649464    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-688696d74-9787c"
Jul 21 12:59:14.652568 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.652538    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5txpn\""
Jul 21 12:59:14.653065 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.653039    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 12:59:14.653551 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.653531    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 12:59:14.660226 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.660207    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-688696d74-9787c"]
Jul 21 12:59:14.752918 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.752822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-webhook-cert\") pod \"maas-controller-688696d74-9787c\" (UID: \"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc\") " pod="opendatahub/maas-controller-688696d74-9787c"
Jul 21 12:59:14.752918 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.752879    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9lbr\" (UniqueName: \"kubernetes.io/projected/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-kube-api-access-m9lbr\") pod \"maas-controller-688696d74-9787c\" (UID: \"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc\") " pod="opendatahub/maas-controller-688696d74-9787c"
Jul 21 12:59:14.853661 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.853617    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-webhook-cert\") pod \"maas-controller-688696d74-9787c\" (UID: \"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc\") " pod="opendatahub/maas-controller-688696d74-9787c"
Jul 21 12:59:14.853844 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.853708    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9lbr\" (UniqueName: \"kubernetes.io/projected/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-kube-api-access-m9lbr\") pod \"maas-controller-688696d74-9787c\" (UID: \"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc\") " pod="opendatahub/maas-controller-688696d74-9787c"
Jul 21 12:59:14.853844 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:59:14.853778    2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 21 12:59:14.853938 ip-10-0-128-74 kubenswrapper[2563]: E0721 12:59:14.853867    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-webhook-cert podName:a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc nodeName:}" failed. No retries permitted until 2026-07-21 12:59:15.353850929 +0000 UTC m=+487.879354063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-webhook-cert") pod "maas-controller-688696d74-9787c" (UID: "a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc") : secret "maas-controller-webhook-cert" not found
Jul 21 12:59:14.863427 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:14.863397    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9lbr\" (UniqueName: \"kubernetes.io/projected/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-kube-api-access-m9lbr\") pod \"maas-controller-688696d74-9787c\" (UID: \"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc\") " pod="opendatahub/maas-controller-688696d74-9787c"
Jul 21 12:59:15.359363 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:15.359325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-webhook-cert\") pod \"maas-controller-688696d74-9787c\" (UID: \"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc\") " pod="opendatahub/maas-controller-688696d74-9787c"
Jul 21 12:59:15.361919 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:15.361898    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-webhook-cert\") pod \"maas-controller-688696d74-9787c\" (UID: \"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc\") " pod="opendatahub/maas-controller-688696d74-9787c"
Jul 21 12:59:15.395699 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:15.395665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:15.560105 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:15.560054    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-688696d74-9787c"
Jul 21 12:59:15.697031 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:15.697002    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-688696d74-9787c"]
Jul 21 12:59:15.699479 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:59:15.699441    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda22b7b8b_71cc_4f7d_8f8d_8ebe7b115adc.slice/crio-8232af524ea29ae047790c399569fe12521617dddd66f97ab30e2d0f9c26decc WatchSource:0}: Error finding container 8232af524ea29ae047790c399569fe12521617dddd66f97ab30e2d0f9c26decc: Status 404 returned error can't find the container with id 8232af524ea29ae047790c399569fe12521617dddd66f97ab30e2d0f9c26decc
Jul 21 12:59:16.058859 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:16.058772    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-688696d74-9787c" event={"ID":"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc","Type":"ContainerStarted","Data":"8232af524ea29ae047790c399569fe12521617dddd66f97ab30e2d0f9c26decc"}
Jul 21 12:59:16.086277 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:16.086233    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:16.189585 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:16.189546    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:18.267989 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:18.267950    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:18.440075 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:18.440043    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:19.073200 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:19.073161    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-688696d74-9787c" event={"ID":"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc","Type":"ContainerStarted","Data":"69af2ea8ec4a67a080c111140de08474afd401480da3be944c793970bf5766cc"}
Jul 21 12:59:19.073395 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:19.073282    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-688696d74-9787c"
Jul 21 12:59:19.102955 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:19.102905    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-688696d74-9787c" podStartSLOduration=2.702735318 podStartE2EDuration="5.102888677s" totalImagesPullingTime="2.400153359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:59:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:59:19.101177343 +0000 UTC m=+491.626680500" watchObservedRunningTime="2026-07-21 12:59:19.102888677 +0000 UTC m=+491.628391832"
Jul 21 12:59:30.080655 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:30.080629    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-688696d74-9787c"
Jul 21 12:59:49.068929 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:49.068873    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:49.576904 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:49.576866    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:49.660152 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:49.660117    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:49.909525 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:49.909493    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:50.123870 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.123840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:50.222387 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.222273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:50.374064 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.374030    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-8xf42"]
Jul 21 12:59:50.383529 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.383498    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-8xf42"
Jul 21 12:59:50.387151 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.387028    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 21 12:59:50.387329 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.387161    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9bfjm\""
Jul 21 12:59:50.388591 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.388565    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-8xf42"]
Jul 21 12:59:50.469588 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.469553    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hxkb\" (UniqueName: \"kubernetes.io/projected/55144294-dceb-4f6b-95fc-921511fc0c19-kube-api-access-5hxkb\") pod \"payload-processing-5bf47cd988-8xf42\" (UID: \"55144294-dceb-4f6b-95fc-921511fc0c19\") " pod="openshift-ingress/payload-processing-5bf47cd988-8xf42"
Jul 21 12:59:50.469784 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.469627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55144294-dceb-4f6b-95fc-921511fc0c19-plugins-config-volume\") pod \"payload-processing-5bf47cd988-8xf42\" (UID: \"55144294-dceb-4f6b-95fc-921511fc0c19\") " pod="openshift-ingress/payload-processing-5bf47cd988-8xf42"
Jul 21 12:59:50.570451 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.570348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55144294-dceb-4f6b-95fc-921511fc0c19-plugins-config-volume\") pod \"payload-processing-5bf47cd988-8xf42\" (UID: \"55144294-dceb-4f6b-95fc-921511fc0c19\") " pod="openshift-ingress/payload-processing-5bf47cd988-8xf42"
Jul 21 12:59:50.570451 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.570428    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hxkb\" (UniqueName: \"kubernetes.io/projected/55144294-dceb-4f6b-95fc-921511fc0c19-kube-api-access-5hxkb\") pod \"payload-processing-5bf47cd988-8xf42\" (UID: \"55144294-dceb-4f6b-95fc-921511fc0c19\") " pod="openshift-ingress/payload-processing-5bf47cd988-8xf42"
Jul 21 12:59:50.571149 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.571128    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55144294-dceb-4f6b-95fc-921511fc0c19-plugins-config-volume\") pod \"payload-processing-5bf47cd988-8xf42\" (UID: \"55144294-dceb-4f6b-95fc-921511fc0c19\") " pod="openshift-ingress/payload-processing-5bf47cd988-8xf42"
Jul 21 12:59:50.594345 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.594315    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hxkb\" (UniqueName: \"kubernetes.io/projected/55144294-dceb-4f6b-95fc-921511fc0c19-kube-api-access-5hxkb\") pod \"payload-processing-5bf47cd988-8xf42\" (UID: \"55144294-dceb-4f6b-95fc-921511fc0c19\") " pod="openshift-ingress/payload-processing-5bf47cd988-8xf42"
Jul 21 12:59:50.698558 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.698519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-8xf42"
Jul 21 12:59:50.864920 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:50.864895    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-8xf42"]
Jul 21 12:59:50.867146 ip-10-0-128-74 kubenswrapper[2563]: W0721 12:59:50.867110    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55144294_dceb_4f6b_95fc_921511fc0c19.slice/crio-720251f0febda836afb5207fa4f31da0dbbe8df5bc4ebbcf8f78870f5298e57a WatchSource:0}: Error finding container 720251f0febda836afb5207fa4f31da0dbbe8df5bc4ebbcf8f78870f5298e57a: Status 404 returned error can't find the container with id 720251f0febda836afb5207fa4f31da0dbbe8df5bc4ebbcf8f78870f5298e57a
Jul 21 12:59:51.218116 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:51.218026    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-8xf42" event={"ID":"55144294-dceb-4f6b-95fc-921511fc0c19","Type":"ContainerStarted","Data":"720251f0febda836afb5207fa4f31da0dbbe8df5bc4ebbcf8f78870f5298e57a"}
Jul 21 12:59:54.232309 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:54.232254    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-8xf42" event={"ID":"55144294-dceb-4f6b-95fc-921511fc0c19","Type":"ContainerStarted","Data":"91383cb4084b5e61795e5fa469f6a19df89338c466d70761c4b9a8f2fc7bc888"}
Jul 21 12:59:54.232771 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:54.232331    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-8xf42"
Jul 21 12:59:54.249404 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:54.249328    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-8xf42" podStartSLOduration=1.284727387 podStartE2EDuration="4.249284195s" totalImagesPullingTime="2.964556808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:59:54.248482988 +0000 UTC m=+526.773986145" watchObservedRunningTime="2026-07-21 12:59:54.249284195 +0000 UTC m=+526.774787352"
Jul 21 12:59:57.318636 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:57.318597    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:58.250750 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:58.250668    2563 generic.go:356] "Generic (PLEG): container finished" podID="ff529716-9b98-4184-b2ef-9ae9e1b3e428" containerID="45116e920c61df7f0c9a4401a8d82d064854341aab57dddd5ef574a4aa037660" exitCode=0
Jul 21 12:59:58.250750 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:58.250709    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb" event={"ID":"ff529716-9b98-4184-b2ef-9ae9e1b3e428","Type":"ContainerDied","Data":"45116e920c61df7f0c9a4401a8d82d064854341aab57dddd5ef574a4aa037660"}
Jul 21 12:59:58.251042 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:58.251031    2563 scope.go:117] "RemoveContainer" containerID="45116e920c61df7f0c9a4401a8d82d064854341aab57dddd5ef574a4aa037660"
Jul 21 12:59:58.251991 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:58.251961    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:59:58.252089 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:58.252030    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:59:58.252089 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:58.252062    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 12:59:58.928510 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:58.928472    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:59.028093 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:59.028051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 12:59:59.256358 ip-10-0-128-74 kubenswrapper[2563]: I0721 12:59:59.256239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb" event={"ID":"ff529716-9b98-4184-b2ef-9ae9e1b3e428","Type":"ContainerStarted","Data":"1e43d3e221bae2d1e9af83188ef24d5caa341e615f56b6d3a62a100a30de5ea8"}
Jul 21 13:00:00.089511 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:00.089482    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 13:00:00.090735 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:00.090696    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-xxfdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Jul 21 13:00:00.090866 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:00.090775    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb" podUID="ff529716-9b98-4184-b2ef-9ae9e1b3e428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Jul 21 13:00:01.090073 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:01.090018    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-xxfdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Jul 21 13:00:01.090536 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:01.090087    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb" podUID="ff529716-9b98-4184-b2ef-9ae9e1b3e428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Jul 21 13:00:02.090115 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:02.090063    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-xxfdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Jul 21 13:00:02.090651 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:02.090153    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb" podUID="ff529716-9b98-4184-b2ef-9ae9e1b3e428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Jul 21 13:00:02.882178 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:02.882135    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 13:00:03.093576 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:03.093530    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-xxfdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Jul 21 13:00:03.094054 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:03.093600    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb" podUID="ff529716-9b98-4184-b2ef-9ae9e1b3e428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Jul 21 13:00:04.092984 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:04.092950    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 13:00:04.094056 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:04.094032    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-xxfdb"
Jul 21 13:00:05.239326 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:05.239271    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-8xf42"
Jul 21 13:00:16.468180 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:16.468143    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:00:26.027022 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:26.026979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:00:30.722778 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.722742    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"]
Jul 21 13:00:30.726000 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.725978    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.728962 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.728939    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 21 13:00:30.729093 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.728981    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dv5rb\""
Jul 21 13:00:30.730070 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.730056    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 21 13:00:30.730345 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.730328    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 21 13:00:30.739705 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.739678    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"]
Jul 21 13:00:30.750664 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.750626    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:00:30.856979 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.856934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d84addf1-9568-4631-8ca9-78fad1357703-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.856979 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.856983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d84addf1-9568-4631-8ca9-78fad1357703-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.857188 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.857004    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d84addf1-9568-4631-8ca9-78fad1357703-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.857188 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.857083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d84addf1-9568-4631-8ca9-78fad1357703-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.857188 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.857114    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d84addf1-9568-4631-8ca9-78fad1357703-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.857188 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.857176    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9rn9\" (UniqueName: \"kubernetes.io/projected/d84addf1-9568-4631-8ca9-78fad1357703-kube-api-access-c9rn9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.958451 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.958411    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d84addf1-9568-4631-8ca9-78fad1357703-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.958451 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.958450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d84addf1-9568-4631-8ca9-78fad1357703-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.958688 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.958494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9rn9\" (UniqueName: \"kubernetes.io/projected/d84addf1-9568-4631-8ca9-78fad1357703-kube-api-access-c9rn9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.958688 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.958575    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d84addf1-9568-4631-8ca9-78fad1357703-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.958688 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.958607    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d84addf1-9568-4631-8ca9-78fad1357703-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.958688 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.958636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d84addf1-9568-4631-8ca9-78fad1357703-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.959004 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.958972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d84addf1-9568-4631-8ca9-78fad1357703-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.959004 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.958971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d84addf1-9568-4631-8ca9-78fad1357703-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.959171 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.959023    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d84addf1-9568-4631-8ca9-78fad1357703-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.960889 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.960865    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d84addf1-9568-4631-8ca9-78fad1357703-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.961240 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.961222    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d84addf1-9568-4631-8ca9-78fad1357703-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:30.969616 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:30.969592    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9rn9\" (UniqueName: \"kubernetes.io/projected/d84addf1-9568-4631-8ca9-78fad1357703-kube-api-access-c9rn9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rkjll\" (UID: \"d84addf1-9568-4631-8ca9-78fad1357703\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:31.036548 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:31.036450    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:31.183741 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:31.183710    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"]
Jul 21 13:00:31.187046 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:00:31.187018    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd84addf1_9568_4631_8ca9_78fad1357703.slice/crio-6e07cd7d8801e54c5a1578a13297cf5c1f3f7083b2fc3e82e9c956ee436157dc WatchSource:0}: Error finding container 6e07cd7d8801e54c5a1578a13297cf5c1f3f7083b2fc3e82e9c956ee436157dc: Status 404 returned error can't find the container with id 6e07cd7d8801e54c5a1578a13297cf5c1f3f7083b2fc3e82e9c956ee436157dc
Jul 21 13:00:31.400210 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:31.400173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll" event={"ID":"d84addf1-9568-4631-8ca9-78fad1357703","Type":"ContainerStarted","Data":"6e07cd7d8801e54c5a1578a13297cf5c1f3f7083b2fc3e82e9c956ee436157dc"}
Jul 21 13:00:37.255470 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:37.255435    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:00:41.444926 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:41.444886    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll" event={"ID":"d84addf1-9568-4631-8ca9-78fad1357703","Type":"ContainerStarted","Data":"10425d91233193d2e03da728a8b4dc2c745c62a1435c6c5cb51cb07f6c0429f8"}
Jul 21 13:00:41.466470 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:41.466412    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll" podStartSLOduration=1.314872584 podStartE2EDuration="11.46639388s" totalImagesPullingTime="10.151521296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:00:41.465170442 +0000 UTC m=+573.990673615" watchObservedRunningTime="2026-07-21 13:00:41.46639388 +0000 UTC m=+573.991897035"
Jul 21 13:00:41.960045 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:41.960007    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:00:45.717070 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.717037    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"]
Jul 21 13:00:45.720592 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.720566    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.723350 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.723326    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 21 13:00:45.732829 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.732806    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"]
Jul 21 13:00:45.756174 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.756142    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:00:45.787033 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.786989    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/150aa186-45f3-42a3-ab16-49a29fdaf457-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.787033 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.787031    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/150aa186-45f3-42a3-ab16-49a29fdaf457-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.787233 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.787046    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/150aa186-45f3-42a3-ab16-49a29fdaf457-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.787233 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.787066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/150aa186-45f3-42a3-ab16-49a29fdaf457-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.787233 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.787187    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jp6q\" (UniqueName: \"kubernetes.io/projected/150aa186-45f3-42a3-ab16-49a29fdaf457-kube-api-access-2jp6q\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.787233 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.787214    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/150aa186-45f3-42a3-ab16-49a29fdaf457-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.888796 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.888751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jp6q\" (UniqueName: \"kubernetes.io/projected/150aa186-45f3-42a3-ab16-49a29fdaf457-kube-api-access-2jp6q\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.889053 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.889022    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/150aa186-45f3-42a3-ab16-49a29fdaf457-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.889185 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.889101    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/150aa186-45f3-42a3-ab16-49a29fdaf457-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.889185 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.889130    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/150aa186-45f3-42a3-ab16-49a29fdaf457-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.889185 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.889154    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/150aa186-45f3-42a3-ab16-49a29fdaf457-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.889185 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.889175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/150aa186-45f3-42a3-ab16-49a29fdaf457-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.889911 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.889883    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/150aa186-45f3-42a3-ab16-49a29fdaf457-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.890529 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.890265    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/150aa186-45f3-42a3-ab16-49a29fdaf457-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.890529 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.890470    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/150aa186-45f3-42a3-ab16-49a29fdaf457-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.892244 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.892224    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/150aa186-45f3-42a3-ab16-49a29fdaf457-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.892543 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.892520    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/150aa186-45f3-42a3-ab16-49a29fdaf457-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:45.898470 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:45.898446    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jp6q\" (UniqueName: \"kubernetes.io/projected/150aa186-45f3-42a3-ab16-49a29fdaf457-kube-api-access-2jp6q\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd68l\" (UID: \"150aa186-45f3-42a3-ab16-49a29fdaf457\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:46.034028 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:46.033945    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:46.171974 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:46.171946    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"]
Jul 21 13:00:46.173860 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:00:46.173828    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod150aa186_45f3_42a3_ab16_49a29fdaf457.slice/crio-449f593a41b5720fc07fa04811c31a91810af1f72ea0ac0f492edf65e23add9f WatchSource:0}: Error finding container 449f593a41b5720fc07fa04811c31a91810af1f72ea0ac0f492edf65e23add9f: Status 404 returned error can't find the container with id 449f593a41b5720fc07fa04811c31a91810af1f72ea0ac0f492edf65e23add9f
Jul 21 13:00:46.466990 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:46.466951    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l" event={"ID":"150aa186-45f3-42a3-ab16-49a29fdaf457","Type":"ContainerStarted","Data":"449f593a41b5720fc07fa04811c31a91810af1f72ea0ac0f492edf65e23add9f"}
Jul 21 13:00:47.474354 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:47.474312    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l" event={"ID":"150aa186-45f3-42a3-ab16-49a29fdaf457","Type":"ContainerStarted","Data":"cd9eccf5825c3b701fb41f579172c8142c40de82cecce48e7f61d426d49dc529"}
Jul 21 13:00:47.497978 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:47.497921    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l" podStartSLOduration=1.821003487 podStartE2EDuration="2.497905789s" totalImagesPullingTime="676.902302ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:00:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:00:47.496540179 +0000 UTC m=+580.022043334" watchObservedRunningTime="2026-07-21 13:00:47.497905789 +0000 UTC m=+580.023409005"
Jul 21 13:00:51.037388 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:51.037346    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:51.037856 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:51.037422    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:51.050214 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:51.050183    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:51.503493 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:51.503458    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rkjll"
Jul 21 13:00:54.852552 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:54.852512    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:00:56.034833 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:56.034797    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:56.035209 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:56.034862    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:56.047348 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:56.047319    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:00:56.524116 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:00:56.524084    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd68l"
Jul 21 13:01:07.940206 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:01:07.940178    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:01:07.940702 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:01:07.940228    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:02:04.728125 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:02:04.728085    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:02:11.842272 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:02:11.842224    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:02:11.937881 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:02:11.937846    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:02:12.972842 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:02:12.972800    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:02:13.074363 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:02:13.074319    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:03:23.836355 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:23.836248    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-688696d74-9787c"]
Jul 21 13:03:23.836984 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:23.836518    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-688696d74-9787c" podUID="a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc" containerName="manager" containerID="cri-o://69af2ea8ec4a67a080c111140de08474afd401480da3be944c793970bf5766cc" gracePeriod=30
Jul 21 13:03:24.095194 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.095132    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-688696d74-9787c"
Jul 21 13:03:24.134395 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.134361    2563 generic.go:356] "Generic (PLEG): container finished" podID="a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc" containerID="69af2ea8ec4a67a080c111140de08474afd401480da3be944c793970bf5766cc" exitCode=0
Jul 21 13:03:24.134579 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.134414    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-688696d74-9787c"
Jul 21 13:03:24.134579 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.134426    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-688696d74-9787c" event={"ID":"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc","Type":"ContainerDied","Data":"69af2ea8ec4a67a080c111140de08474afd401480da3be944c793970bf5766cc"}
Jul 21 13:03:24.134579 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.134468    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-688696d74-9787c" event={"ID":"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc","Type":"ContainerDied","Data":"8232af524ea29ae047790c399569fe12521617dddd66f97ab30e2d0f9c26decc"}
Jul 21 13:03:24.134579 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.134489    2563 scope.go:117] "RemoveContainer" containerID="69af2ea8ec4a67a080c111140de08474afd401480da3be944c793970bf5766cc"
Jul 21 13:03:24.143786 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.143763    2563 scope.go:117] "RemoveContainer" containerID="69af2ea8ec4a67a080c111140de08474afd401480da3be944c793970bf5766cc"
Jul 21 13:03:24.144123 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:03:24.144101    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69af2ea8ec4a67a080c111140de08474afd401480da3be944c793970bf5766cc\": container with ID starting with 69af2ea8ec4a67a080c111140de08474afd401480da3be944c793970bf5766cc not found: ID does not exist" containerID="69af2ea8ec4a67a080c111140de08474afd401480da3be944c793970bf5766cc"
Jul 21 13:03:24.144234 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.144131    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69af2ea8ec4a67a080c111140de08474afd401480da3be944c793970bf5766cc"} err="failed to get container status \"69af2ea8ec4a67a080c111140de08474afd401480da3be944c793970bf5766cc\": rpc error: code = NotFound desc = could not find container \"69af2ea8ec4a67a080c111140de08474afd401480da3be944c793970bf5766cc\": container with ID starting with 69af2ea8ec4a67a080c111140de08474afd401480da3be944c793970bf5766cc not found: ID does not exist"
Jul 21 13:03:24.205615 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.205577    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-webhook-cert\") pod \"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc\" (UID: \"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc\") "
Jul 21 13:03:24.205819 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.205671    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9lbr\" (UniqueName: \"kubernetes.io/projected/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-kube-api-access-m9lbr\") pod \"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc\" (UID: \"a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc\") "
Jul 21 13:03:24.207894 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.207865    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc" (UID: "a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:03:24.208002 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.207950    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-kube-api-access-m9lbr" (OuterVolumeSpecName: "kube-api-access-m9lbr") pod "a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc" (UID: "a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc"). InnerVolumeSpecName "kube-api-access-m9lbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:03:24.307064 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.307029    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-webhook-cert\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:03:24.307064 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.307060    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9lbr\" (UniqueName: \"kubernetes.io/projected/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc-kube-api-access-m9lbr\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:03:24.457948 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.457901    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-688696d74-9787c"]
Jul 21 13:03:24.465948 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:24.465889    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-688696d74-9787c"]
Jul 21 13:03:25.994973 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:03:25.994938    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc" path="/var/lib/kubelet/pods/a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc/volumes"
Jul 21 13:04:20.817858 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:04:20.817804    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:04:20.877140 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:04:20.877065    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:04:43.121873 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:04:43.121787    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:04:44.704465 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:04:44.704435    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:04:54.914815 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:04:54.914778    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:04:55.015103 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:04:55.015067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:05:20.063931 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:05:20.063892    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:05:20.161808 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:05:20.161765    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:05:20.260424 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:05:20.260389    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:05:26.016426 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:05:26.016386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:05:26.089991 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:05:26.089955    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:05:26.209844 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:05:26.209810    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:05:26.316898 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:05:26.316811    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:05:26.465338 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:05:26.465275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:05:32.469583 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:05:32.469526    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:05:32.573190 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:05:32.573160    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:06:07.964774 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:06:07.964693    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:06:07.965534 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:06:07.965515    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:08:38.244515 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:08:38.244482    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:08:40.266888 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:08:40.266852    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:09:55.880226 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:09:55.880184    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:10:43.858661 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:10:43.858577    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:11:07.994858 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:11:07.994820    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:11:07.997499 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:11:07.997480    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:13:29.607432 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.607337    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"]
Jul 21 13:13:29.607939 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.607766    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc" containerName="manager"
Jul 21 13:13:29.607939 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.607784    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc" containerName="manager"
Jul 21 13:13:29.607939 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.607857    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a22b7b8b-71cc-4f7d-8f8d-8ebe7b115adc" containerName="manager"
Jul 21 13:13:29.611105 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.611083    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.613823 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.613800    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-35ceff91-kserve-self-signed-certs\""
Jul 21 13:13:29.630450 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.630418    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"]
Jul 21 13:13:29.681901 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.681863    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:13:29.708669 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.708633    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/085d3a02-d031-4c07-ae81-b4f897ca7310-tls-certs\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.708669 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.708672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-dshm\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.708919 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.708712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-home\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.708919 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.708820    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk92k\" (UniqueName: \"kubernetes.io/projected/085d3a02-d031-4c07-ae81-b4f897ca7310-kube-api-access-hk92k\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.708919 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.708872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-model-cache\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.709045 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.708942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-tmp-dir\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.809820 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.809784    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-tmp-dir\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.809820 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.809838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/085d3a02-d031-4c07-ae81-b4f897ca7310-tls-certs\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.810064 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.809856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-dshm\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.810064 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.809980    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-home\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.810064 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.810048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk92k\" (UniqueName: \"kubernetes.io/projected/085d3a02-d031-4c07-ae81-b4f897ca7310-kube-api-access-hk92k\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.810218 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.810098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-model-cache\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.810360 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.810333    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-tmp-dir\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.810486 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.810398    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-home\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.810486 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.810415    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-model-cache\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.812163 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.812143    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-dshm\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.812438 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.812423    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/085d3a02-d031-4c07-ae81-b4f897ca7310-tls-certs\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.818987 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.818951    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk92k\" (UniqueName: \"kubernetes.io/projected/085d3a02-d031-4c07-ae81-b4f897ca7310-kube-api-access-hk92k\") pod \"e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") " pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:29.920202 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:29.920094    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:30.065197 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:30.065169    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"]
Jul 21 13:13:30.067233 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:13:30.067208    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod085d3a02_d031_4c07_ae81_b4f897ca7310.slice/crio-3aaaf2f40da7ca161e0076d5de605e3a36ea9877b8ea93a1cc316d1b67b7213e WatchSource:0}: Error finding container 3aaaf2f40da7ca161e0076d5de605e3a36ea9877b8ea93a1cc316d1b67b7213e: Status 404 returned error can't find the container with id 3aaaf2f40da7ca161e0076d5de605e3a36ea9877b8ea93a1cc316d1b67b7213e
Jul 21 13:13:30.738037 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:30.738001    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt" event={"ID":"085d3a02-d031-4c07-ae81-b4f897ca7310","Type":"ContainerStarted","Data":"63b456444f48fcb9ffba5a9db4e16ef4c76217db41a34f3a378701014d90ab03"}
Jul 21 13:13:30.738454 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:30.738044    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt" event={"ID":"085d3a02-d031-4c07-ae81-b4f897ca7310","Type":"ContainerStarted","Data":"3aaaf2f40da7ca161e0076d5de605e3a36ea9877b8ea93a1cc316d1b67b7213e"}
Jul 21 13:13:39.175367 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:39.175329    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:13:39.920556 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:39.920515    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:39.920726 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:39.920573    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:39.933230 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:39.933205    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:13:39.953645 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:39.953592    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt" podStartSLOduration=10.953575047 podStartE2EDuration="10.953575047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:13:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:13:30.763262882 +0000 UTC m=+1343.288766038" watchObservedRunningTime="2026-07-21 13:13:39.953575047 +0000 UTC m=+1352.479078201"
Jul 21 13:13:40.791547 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:13:40.791514    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:14:22.047117 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:22.047080    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"]
Jul 21 13:14:22.074930 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:22.074901    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:14:23.450109 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:23.450075    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:14:37.066903 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.066835    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt" podUID="085d3a02-d031-4c07-ae81-b4f897ca7310" containerName="main" containerID="cri-o://63b456444f48fcb9ffba5a9db4e16ef4c76217db41a34f3a378701014d90ab03" gracePeriod=45
Jul 21 13:14:37.317909 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.317847    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:14:37.421062 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.421031    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-home\") pod \"085d3a02-d031-4c07-ae81-b4f897ca7310\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") "
Jul 21 13:14:37.421252 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.421077    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/085d3a02-d031-4c07-ae81-b4f897ca7310-tls-certs\") pod \"085d3a02-d031-4c07-ae81-b4f897ca7310\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") "
Jul 21 13:14:37.421252 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.421119    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-tmp-dir\") pod \"085d3a02-d031-4c07-ae81-b4f897ca7310\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") "
Jul 21 13:14:37.421252 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.421161    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-dshm\") pod \"085d3a02-d031-4c07-ae81-b4f897ca7310\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") "
Jul 21 13:14:37.421252 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.421197    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-model-cache\") pod \"085d3a02-d031-4c07-ae81-b4f897ca7310\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") "
Jul 21 13:14:37.421252 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.421237    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk92k\" (UniqueName: \"kubernetes.io/projected/085d3a02-d031-4c07-ae81-b4f897ca7310-kube-api-access-hk92k\") pod \"085d3a02-d031-4c07-ae81-b4f897ca7310\" (UID: \"085d3a02-d031-4c07-ae81-b4f897ca7310\") "
Jul 21 13:14:37.421498 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.421334    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-home" (OuterVolumeSpecName: "home") pod "085d3a02-d031-4c07-ae81-b4f897ca7310" (UID: "085d3a02-d031-4c07-ae81-b4f897ca7310"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:14:37.421498 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.421431    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "085d3a02-d031-4c07-ae81-b4f897ca7310" (UID: "085d3a02-d031-4c07-ae81-b4f897ca7310"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:14:37.421498 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.421481    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-model-cache" (OuterVolumeSpecName: "model-cache") pod "085d3a02-d031-4c07-ae81-b4f897ca7310" (UID: "085d3a02-d031-4c07-ae81-b4f897ca7310"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:14:37.421636 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.421612    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-home\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:14:37.421636 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.421629    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-tmp-dir\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:14:37.421731 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.421638    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-model-cache\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:14:37.423546 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.423521    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/085d3a02-d031-4c07-ae81-b4f897ca7310-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "085d3a02-d031-4c07-ae81-b4f897ca7310" (UID: "085d3a02-d031-4c07-ae81-b4f897ca7310"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:14:37.423546 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.423523    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-dshm" (OuterVolumeSpecName: "dshm") pod "085d3a02-d031-4c07-ae81-b4f897ca7310" (UID: "085d3a02-d031-4c07-ae81-b4f897ca7310"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:14:37.423665 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.423563    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/085d3a02-d031-4c07-ae81-b4f897ca7310-kube-api-access-hk92k" (OuterVolumeSpecName: "kube-api-access-hk92k") pod "085d3a02-d031-4c07-ae81-b4f897ca7310" (UID: "085d3a02-d031-4c07-ae81-b4f897ca7310"). InnerVolumeSpecName "kube-api-access-hk92k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:14:37.522852 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.522818    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/085d3a02-d031-4c07-ae81-b4f897ca7310-dshm\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:14:37.522852 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.522847    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk92k\" (UniqueName: \"kubernetes.io/projected/085d3a02-d031-4c07-ae81-b4f897ca7310-kube-api-access-hk92k\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:14:37.522852 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:37.522858    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/085d3a02-d031-4c07-ae81-b4f897ca7310-tls-certs\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:14:38.023994 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:38.023958    2563 generic.go:356] "Generic (PLEG): container finished" podID="085d3a02-d031-4c07-ae81-b4f897ca7310" containerID="63b456444f48fcb9ffba5a9db4e16ef4c76217db41a34f3a378701014d90ab03" exitCode=0
Jul 21 13:14:38.024147 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:38.023999    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"
Jul 21 13:14:38.024147 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:38.024038    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt" event={"ID":"085d3a02-d031-4c07-ae81-b4f897ca7310","Type":"ContainerDied","Data":"63b456444f48fcb9ffba5a9db4e16ef4c76217db41a34f3a378701014d90ab03"}
Jul 21 13:14:38.024147 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:38.024079    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt" event={"ID":"085d3a02-d031-4c07-ae81-b4f897ca7310","Type":"ContainerDied","Data":"3aaaf2f40da7ca161e0076d5de605e3a36ea9877b8ea93a1cc316d1b67b7213e"}
Jul 21 13:14:38.024147 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:38.024097    2563 scope.go:117] "RemoveContainer" containerID="63b456444f48fcb9ffba5a9db4e16ef4c76217db41a34f3a378701014d90ab03"
Jul 21 13:14:38.032173 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:38.032155    2563 scope.go:117] "RemoveContainer" containerID="63b456444f48fcb9ffba5a9db4e16ef4c76217db41a34f3a378701014d90ab03"
Jul 21 13:14:38.032527 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:14:38.032505    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63b456444f48fcb9ffba5a9db4e16ef4c76217db41a34f3a378701014d90ab03\": container with ID starting with 63b456444f48fcb9ffba5a9db4e16ef4c76217db41a34f3a378701014d90ab03 not found: ID does not exist" containerID="63b456444f48fcb9ffba5a9db4e16ef4c76217db41a34f3a378701014d90ab03"
Jul 21 13:14:38.032618 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:38.032535    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b456444f48fcb9ffba5a9db4e16ef4c76217db41a34f3a378701014d90ab03"} err="failed to get container status \"63b456444f48fcb9ffba5a9db4e16ef4c76217db41a34f3a378701014d90ab03\": rpc error: code = NotFound desc = could not find container \"63b456444f48fcb9ffba5a9db4e16ef4c76217db41a34f3a378701014d90ab03\": container with ID starting with 63b456444f48fcb9ffba5a9db4e16ef4c76217db41a34f3a378701014d90ab03 not found: ID does not exist"
Jul 21 13:14:38.041902 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:38.041881    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"]
Jul 21 13:14:38.047827 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:38.047805    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-35ceff91-kserve-6c567c5fc5-z4dkt"]
Jul 21 13:14:39.994993 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:14:39.994959    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="085d3a02-d031-4c07-ae81-b4f897ca7310" path="/var/lib/kubelet/pods/085d3a02-d031-4c07-ae81-b4f897ca7310/volumes"
Jul 21 13:16:08.020179 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:16:08.020138    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:16:08.025623 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:16:08.025600    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:17:01.628351 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:01.628256    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:01.721838 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:01.721802    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:37.611113 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:37.611075    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:37.708590 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:37.708525    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:38.078055 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:38.077969    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:43.932140 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:43.932106    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:44.032646 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.032609    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:44.152676 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.152645    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:44.424909 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.424848    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"]
Jul 21 13:17:44.425272 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.425255    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="085d3a02-d031-4c07-ae81-b4f897ca7310" containerName="main"
Jul 21 13:17:44.425397 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.425276    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="085d3a02-d031-4c07-ae81-b4f897ca7310" containerName="main"
Jul 21 13:17:44.425397 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.425388    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="085d3a02-d031-4c07-ae81-b4f897ca7310" containerName="main"
Jul 21 13:17:44.428313 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.428273    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.430920 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.430897    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-209ca517-openshift-default-dockercfg-zzwbv\""
Jul 21 13:17:44.449907 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.449805    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:44.451548 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.451519    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"]
Jul 21 13:17:44.546541 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.546485    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:44.560788 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.560758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-workload-socket\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.560929 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.560794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-podinfo\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.560929 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.560864    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-data\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.560929 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.560900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-envoy\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.561059 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.560949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-credential-socket\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.561059 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.560964    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4xzz\" (UniqueName: \"kubernetes.io/projected/68aaf89b-50f4-497c-ab1f-2144c757fc70-kube-api-access-l4xzz\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.561059 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.561000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68aaf89b-50f4-497c-ab1f-2144c757fc70-istiod-ca-cert\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.561059 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.561019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-token\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.561195 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.561040    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-workload-certs\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.661637 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.661599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-data\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.661840 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.661644    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-envoy\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.661840 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.661698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-credential-socket\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.661840 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.661715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4xzz\" (UniqueName: \"kubernetes.io/projected/68aaf89b-50f4-497c-ab1f-2144c757fc70-kube-api-access-l4xzz\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.661840 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.661735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68aaf89b-50f4-497c-ab1f-2144c757fc70-istiod-ca-cert\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.661840 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.661754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-token\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.662097 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.661855    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-workload-certs\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.662097 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.662070    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-workload-socket\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.662097 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.662090    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-data\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.662243 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.662109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-podinfo\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.662243 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.662159    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-credential-socket\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.662243 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.662199    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-workload-certs\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.662381 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.662365    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-workload-socket\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.662524 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.662497    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68aaf89b-50f4-497c-ab1f-2144c757fc70-istiod-ca-cert\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.664217 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.664192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-envoy\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.664614 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.664588    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-podinfo\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.671956 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.671933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-token\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.672146 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.672122    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4xzz\" (UniqueName: \"kubernetes.io/projected/68aaf89b-50f4-497c-ab1f-2144c757fc70-kube-api-access-l4xzz\") pod \"e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") " pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.738262 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.738171    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:44.887108 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.887077    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"]
Jul 21 13:17:44.889809 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:17:44.889783    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68aaf89b_50f4_497c_ab1f_2144c757fc70.slice/crio-7ab8e1f6ada84235fd3dc1ee3d8f48fc428cc26c9af5a1abf9ae1c6b0688c3c0 WatchSource:0}: Error finding container 7ab8e1f6ada84235fd3dc1ee3d8f48fc428cc26c9af5a1abf9ae1c6b0688c3c0: Status 404 returned error can't find the container with id 7ab8e1f6ada84235fd3dc1ee3d8f48fc428cc26c9af5a1abf9ae1c6b0688c3c0
Jul 21 13:17:44.891935 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.891900    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:17:44.891992 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.891969    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:17:44.892029 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:44.891995    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:17:45.806502 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:45.806466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7" event={"ID":"68aaf89b-50f4-497c-ab1f-2144c757fc70","Type":"ContainerStarted","Data":"da2739fed5522647c78c892a4067b7e85ed22618912e0cab42f64017a4e34a3a"}
Jul 21 13:17:45.806938 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:45.806518    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7" event={"ID":"68aaf89b-50f4-497c-ab1f-2144c757fc70","Type":"ContainerStarted","Data":"7ab8e1f6ada84235fd3dc1ee3d8f48fc428cc26c9af5a1abf9ae1c6b0688c3c0"}
Jul 21 13:17:45.829247 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:45.829193    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7" podStartSLOduration=1.829177032 podStartE2EDuration="1.829177032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:17:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:17:45.827268684 +0000 UTC m=+1598.352771839" watchObservedRunningTime="2026-07-21 13:17:45.829177032 +0000 UTC m=+1598.354680188"
Jul 21 13:17:46.739393 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:46.739353    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:46.744376 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:46.744350    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:46.811424 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:46.811388    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:46.812426 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:46.812401    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:17:47.282250 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:47.282209    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:47.452533 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:47.452502    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:47.578052 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:47.577946    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:47.683881 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:47.683852    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:48.238797 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:48.238759    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:17:48.334187 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:17:48.334150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:02.177136 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:02.176682    2563 patch_prober.go:28] interesting pod/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": read tcp 10.133.0.2:45814->10.133.0.33:15021: read: connection reset by peer" start-of-body=
Jul 21 13:18:02.177136 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:02.176746    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7" podUID="68aaf89b-50f4-497c-ab1f-2144c757fc70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": read tcp 10.133.0.2:45814->10.133.0.33:15021: read: connection reset by peer"
Jul 21 13:18:02.881437 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:02.881400    2563 generic.go:356] "Generic (PLEG): container finished" podID="68aaf89b-50f4-497c-ab1f-2144c757fc70" containerID="da2739fed5522647c78c892a4067b7e85ed22618912e0cab42f64017a4e34a3a" exitCode=0
Jul 21 13:18:02.881606 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:02.881469    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7" event={"ID":"68aaf89b-50f4-497c-ab1f-2144c757fc70","Type":"ContainerDied","Data":"da2739fed5522647c78c892a4067b7e85ed22618912e0cab42f64017a4e34a3a"}
Jul 21 13:18:02.881904 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:02.881869    2563 scope.go:117] "RemoveContainer" containerID="da2739fed5522647c78c892a4067b7e85ed22618912e0cab42f64017a4e34a3a"
Jul 21 13:18:02.882886 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:02.882861    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:18:02.882962 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:02.882950    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:18:02.882998 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:02.882982    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:18:03.891862 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:03.891813    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7" event={"ID":"68aaf89b-50f4-497c-ab1f-2144c757fc70","Type":"ContainerStarted","Data":"1ee0028d53c682f3ac743f3a276207b6f4af2cb23b345975e1caf9f4c4a9bd3b"}
Jul 21 13:18:04.738565 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:04.738524    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:18:04.738806 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:04.738780    2563 patch_prober.go:28] interesting pod/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 21 13:18:04.738888 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:04.738837    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7" podUID="68aaf89b-50f4-497c-ab1f-2144c757fc70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 21 13:18:05.738654 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:05.738613    2563 patch_prober.go:28] interesting pod/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 21 13:18:05.739133 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:05.738680    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7" podUID="68aaf89b-50f4-497c-ab1f-2144c757fc70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 21 13:18:06.739150 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:06.739107    2563 patch_prober.go:28] interesting pod/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 21 13:18:06.739561 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:06.739176    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7" podUID="68aaf89b-50f4-497c-ab1f-2144c757fc70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 21 13:18:07.748436 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:07.748403    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:18:07.748934 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:07.748685    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:18:07.749576 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:07.749554    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:18:23.351457 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:23.351421    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:27.982371 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:27.982337    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:28.111049 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:28.111016    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"]
Jul 21 13:18:28.111321 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:28.111254    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7" podUID="68aaf89b-50f4-497c-ab1f-2144c757fc70" containerName="istio-proxy" containerID="cri-o://1ee0028d53c682f3ac743f3a276207b6f4af2cb23b345975e1caf9f4c4a9bd3b" gracePeriod=30
Jul 21 13:18:28.146805 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:28.146777    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:28.403412 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:28.403381    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:29.063975 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.063939    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:29.168777 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.168748    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:29.276013 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.275980    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"]
Jul 21 13:18:29.279424 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.279406    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.282619 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.282594    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-8bfb8415-gw-openshift-default-dockercfg-gw9x4\""
Jul 21 13:18:29.302067 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.302037    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"]
Jul 21 13:18:29.344604 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.344513    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:29.383427 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.383390    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istiod-ca-cert\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.383660 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.383488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xw79\" (UniqueName: \"kubernetes.io/projected/c3da855f-ddea-43d5-a63f-b1632c5cd45a-kube-api-access-9xw79\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.383660 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.383532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-podinfo\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.383660 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.383584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-workload-socket\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.383660 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.383610    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-token\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.383933 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.383660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-envoy\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.383933 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.383726    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-data\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.383933 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.383804    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-credential-socket\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.383933 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.383833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-workload-certs\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.485137 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.485097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-credential-socket\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.485137 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.485136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-workload-certs\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.485436 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.485311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istiod-ca-cert\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.485436 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.485352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xw79\" (UniqueName: \"kubernetes.io/projected/c3da855f-ddea-43d5-a63f-b1632c5cd45a-kube-api-access-9xw79\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.485436 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.485387    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-podinfo\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.486036 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.485452    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-workload-socket\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.486036 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.485477    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-token\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.486036 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.485529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-envoy\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.486036 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.485561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-data\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.486036 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.485584    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-workload-certs\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.486036 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.485622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-credential-socket\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.486036 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.485857    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-data\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.486036 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.485971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-workload-socket\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.486036 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.486024    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istiod-ca-cert\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.488437 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.488415    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-envoy\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.488630 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.488610    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-podinfo\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.495309 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.495273    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-token\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.495489 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.495453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xw79\" (UniqueName: \"kubernetes.io/projected/c3da855f-ddea-43d5-a63f-b1632c5cd45a-kube-api-access-9xw79\") pod \"e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") " pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.590503 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.590453    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:29.746250 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.746217    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"]
Jul 21 13:18:29.747556 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:18:29.747524    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3da855f_ddea_43d5_a63f_b1632c5cd45a.slice/crio-d83092de4c7db98b23a50b5c3786923ff86f55484d8ca14040b4db9c1b2a7b6c WatchSource:0}: Error finding container d83092de4c7db98b23a50b5c3786923ff86f55484d8ca14040b4db9c1b2a7b6c: Status 404 returned error can't find the container with id d83092de4c7db98b23a50b5c3786923ff86f55484d8ca14040b4db9c1b2a7b6c
Jul 21 13:18:29.749699 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.749653    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:18:29.749772 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.749744    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:18:29.749815 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:29.749787    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:18:30.012970 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:30.012915    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf" event={"ID":"c3da855f-ddea-43d5-a63f-b1632c5cd45a","Type":"ContainerStarted","Data":"da26be29de20303ed527f2a0d490d44c098c11a799ab1311e6369e58151ae8d5"}
Jul 21 13:18:30.013171 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:30.012979    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf" event={"ID":"c3da855f-ddea-43d5-a63f-b1632c5cd45a","Type":"ContainerStarted","Data":"d83092de4c7db98b23a50b5c3786923ff86f55484d8ca14040b4db9c1b2a7b6c"}
Jul 21 13:18:30.034741 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:30.034709    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:30.055157 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:30.055102    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf" podStartSLOduration=1.055086032 podStartE2EDuration="1.055086032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:18:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:18:30.052974521 +0000 UTC m=+1642.578477677" watchObservedRunningTime="2026-07-21 13:18:30.055086032 +0000 UTC m=+1642.580589187"
Jul 21 13:18:30.591573 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:30.591536    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:30.596498 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:30.596471    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:31.018130 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:31.018096    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:31.019008 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:31.018991    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:31.622176 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:31.622144    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:31.787332 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:31.787275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:31.932979 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:31.932891    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:32.328694 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:32.328606    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:32.429736 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:32.429700    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:33.371910 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.371845    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:18:33.524884 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.524839    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-envoy\") pod \"68aaf89b-50f4-497c-ab1f-2144c757fc70\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") "
Jul 21 13:18:33.524884 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.524881    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68aaf89b-50f4-497c-ab1f-2144c757fc70-istiod-ca-cert\") pod \"68aaf89b-50f4-497c-ab1f-2144c757fc70\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") "
Jul 21 13:18:33.525139 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.524915    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4xzz\" (UniqueName: \"kubernetes.io/projected/68aaf89b-50f4-497c-ab1f-2144c757fc70-kube-api-access-l4xzz\") pod \"68aaf89b-50f4-497c-ab1f-2144c757fc70\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") "
Jul 21 13:18:33.525139 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525040    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-workload-socket\") pod \"68aaf89b-50f4-497c-ab1f-2144c757fc70\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") "
Jul 21 13:18:33.525139 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525096    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-podinfo\") pod \"68aaf89b-50f4-497c-ab1f-2144c757fc70\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") "
Jul 21 13:18:33.525342 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525142    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-token\") pod \"68aaf89b-50f4-497c-ab1f-2144c757fc70\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") "
Jul 21 13:18:33.525342 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525177    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-workload-certs\") pod \"68aaf89b-50f4-497c-ab1f-2144c757fc70\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") "
Jul 21 13:18:33.525342 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525209    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-data\") pod \"68aaf89b-50f4-497c-ab1f-2144c757fc70\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") "
Jul 21 13:18:33.525342 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525252    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-credential-socket\") pod \"68aaf89b-50f4-497c-ab1f-2144c757fc70\" (UID: \"68aaf89b-50f4-497c-ab1f-2144c757fc70\") "
Jul 21 13:18:33.525568 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525368    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "68aaf89b-50f4-497c-ab1f-2144c757fc70" (UID: "68aaf89b-50f4-497c-ab1f-2144c757fc70"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:18:33.525568 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525380    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68aaf89b-50f4-497c-ab1f-2144c757fc70-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "68aaf89b-50f4-497c-ab1f-2144c757fc70" (UID: "68aaf89b-50f4-497c-ab1f-2144c757fc70"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:18:33.525674 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525587    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "68aaf89b-50f4-497c-ab1f-2144c757fc70" (UID: "68aaf89b-50f4-497c-ab1f-2144c757fc70"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:18:33.525674 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525624    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "68aaf89b-50f4-497c-ab1f-2144c757fc70" (UID: "68aaf89b-50f4-497c-ab1f-2144c757fc70"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:18:33.525831 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525742    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-workload-socket\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.525831 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525760    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-workload-certs\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.525831 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525773    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-credential-socket\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.525831 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.525791    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68aaf89b-50f4-497c-ab1f-2144c757fc70-istiod-ca-cert\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.526540 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.526514    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-data" (OuterVolumeSpecName: "istio-data") pod "68aaf89b-50f4-497c-ab1f-2144c757fc70" (UID: "68aaf89b-50f4-497c-ab1f-2144c757fc70"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:18:33.527534 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.527500    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "68aaf89b-50f4-497c-ab1f-2144c757fc70" (UID: "68aaf89b-50f4-497c-ab1f-2144c757fc70"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:18:33.527534 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.527521    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68aaf89b-50f4-497c-ab1f-2144c757fc70-kube-api-access-l4xzz" (OuterVolumeSpecName: "kube-api-access-l4xzz") pod "68aaf89b-50f4-497c-ab1f-2144c757fc70" (UID: "68aaf89b-50f4-497c-ab1f-2144c757fc70"). InnerVolumeSpecName "kube-api-access-l4xzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:18:33.527901 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.527875    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-token" (OuterVolumeSpecName: "istio-token") pod "68aaf89b-50f4-497c-ab1f-2144c757fc70" (UID: "68aaf89b-50f4-497c-ab1f-2144c757fc70"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:18:33.527901 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.527878    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "68aaf89b-50f4-497c-ab1f-2144c757fc70" (UID: "68aaf89b-50f4-497c-ab1f-2144c757fc70"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:18:33.626464 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.626371    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-envoy\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.626464 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.626406    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4xzz\" (UniqueName: \"kubernetes.io/projected/68aaf89b-50f4-497c-ab1f-2144c757fc70-kube-api-access-l4xzz\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.626464 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.626416    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-podinfo\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.626464 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.626427    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-token\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.626464 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:33.626435    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68aaf89b-50f4-497c-ab1f-2144c757fc70-istio-data\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:18:34.032247 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:34.032210    2563 generic.go:356] "Generic (PLEG): container finished" podID="68aaf89b-50f4-497c-ab1f-2144c757fc70" containerID="1ee0028d53c682f3ac743f3a276207b6f4af2cb23b345975e1caf9f4c4a9bd3b" exitCode=0
Jul 21 13:18:34.032464 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:34.032255    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"
Jul 21 13:18:34.032464 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:34.032316    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7" event={"ID":"68aaf89b-50f4-497c-ab1f-2144c757fc70","Type":"ContainerDied","Data":"1ee0028d53c682f3ac743f3a276207b6f4af2cb23b345975e1caf9f4c4a9bd3b"}
Jul 21 13:18:34.032464 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:34.032382    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7" event={"ID":"68aaf89b-50f4-497c-ab1f-2144c757fc70","Type":"ContainerDied","Data":"7ab8e1f6ada84235fd3dc1ee3d8f48fc428cc26c9af5a1abf9ae1c6b0688c3c0"}
Jul 21 13:18:34.032464 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:34.032416    2563 scope.go:117] "RemoveContainer" containerID="1ee0028d53c682f3ac743f3a276207b6f4af2cb23b345975e1caf9f4c4a9bd3b"
Jul 21 13:18:34.041278 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:34.041261    2563 scope.go:117] "RemoveContainer" containerID="da2739fed5522647c78c892a4067b7e85ed22618912e0cab42f64017a4e34a3a"
Jul 21 13:18:34.049591 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:34.049572    2563 scope.go:117] "RemoveContainer" containerID="1ee0028d53c682f3ac743f3a276207b6f4af2cb23b345975e1caf9f4c4a9bd3b"
Jul 21 13:18:34.049896 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:18:34.049874    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ee0028d53c682f3ac743f3a276207b6f4af2cb23b345975e1caf9f4c4a9bd3b\": container with ID starting with 1ee0028d53c682f3ac743f3a276207b6f4af2cb23b345975e1caf9f4c4a9bd3b not found: ID does not exist" containerID="1ee0028d53c682f3ac743f3a276207b6f4af2cb23b345975e1caf9f4c4a9bd3b"
Jul 21 13:18:34.049967 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:34.049905    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ee0028d53c682f3ac743f3a276207b6f4af2cb23b345975e1caf9f4c4a9bd3b"} err="failed to get container status \"1ee0028d53c682f3ac743f3a276207b6f4af2cb23b345975e1caf9f4c4a9bd3b\": rpc error: code = NotFound desc = could not find container \"1ee0028d53c682f3ac743f3a276207b6f4af2cb23b345975e1caf9f4c4a9bd3b\": container with ID starting with 1ee0028d53c682f3ac743f3a276207b6f4af2cb23b345975e1caf9f4c4a9bd3b not found: ID does not exist"
Jul 21 13:18:34.049967 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:34.049923    2563 scope.go:117] "RemoveContainer" containerID="da2739fed5522647c78c892a4067b7e85ed22618912e0cab42f64017a4e34a3a"
Jul 21 13:18:34.050171 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:18:34.050154    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da2739fed5522647c78c892a4067b7e85ed22618912e0cab42f64017a4e34a3a\": container with ID starting with da2739fed5522647c78c892a4067b7e85ed22618912e0cab42f64017a4e34a3a not found: ID does not exist" containerID="da2739fed5522647c78c892a4067b7e85ed22618912e0cab42f64017a4e34a3a"
Jul 21 13:18:34.050251 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:34.050175    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da2739fed5522647c78c892a4067b7e85ed22618912e0cab42f64017a4e34a3a"} err="failed to get container status \"da2739fed5522647c78c892a4067b7e85ed22618912e0cab42f64017a4e34a3a\": rpc error: code = NotFound desc = could not find container \"da2739fed5522647c78c892a4067b7e85ed22618912e0cab42f64017a4e34a3a\": container with ID starting with da2739fed5522647c78c892a4067b7e85ed22618912e0cab42f64017a4e34a3a not found: ID does not exist"
Jul 21 13:18:34.053164 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:34.053139    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"]
Jul 21 13:18:34.058828 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:34.058803    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-209ca517-openshift-default-6496fc8f7f-9jzt7"]
Jul 21 13:18:35.995859 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:35.995822    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68aaf89b-50f4-497c-ab1f-2144c757fc70" path="/var/lib/kubelet/pods/68aaf89b-50f4-497c-ab1f-2144c757fc70/volumes"
Jul 21 13:18:43.850517 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:43.850483    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:18:46.453729 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:46.452782    2563 patch_prober.go:28] interesting pod/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": read tcp 10.133.0.2:43470->10.133.0.34:15021: read: connection reset by peer" start-of-body=
Jul 21 13:18:46.453729 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:46.452880    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf" podUID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": read tcp 10.133.0.2:43470->10.133.0.34:15021: read: connection reset by peer"
Jul 21 13:18:47.097380 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:47.097345    2563 generic.go:356] "Generic (PLEG): container finished" podID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerID="da26be29de20303ed527f2a0d490d44c098c11a799ab1311e6369e58151ae8d5" exitCode=0
Jul 21 13:18:47.097577 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:47.097386    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf" event={"ID":"c3da855f-ddea-43d5-a63f-b1632c5cd45a","Type":"ContainerDied","Data":"da26be29de20303ed527f2a0d490d44c098c11a799ab1311e6369e58151ae8d5"}
Jul 21 13:18:47.097756 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:47.097744    2563 scope.go:117] "RemoveContainer" containerID="da26be29de20303ed527f2a0d490d44c098c11a799ab1311e6369e58151ae8d5"
Jul 21 13:18:47.098823 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:47.098794    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:18:47.098882 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:47.098862    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:18:47.098920 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:47.098904    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:18:48.103222 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:48.103182    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf" event={"ID":"c3da855f-ddea-43d5-a63f-b1632c5cd45a","Type":"ContainerStarted","Data":"e56ada89169c401faa6d37605ee718e7b0e98e4ee283bbb3261762eb3cc9cafc"}
Jul 21 13:18:48.591735 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:48.591269    2563 patch_prober.go:28] interesting pod/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 21 13:18:48.591735 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:48.591374    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf" podUID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 21 13:18:48.592093 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:48.592012    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:49.591220 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:49.591170    2563 patch_prober.go:28] interesting pod/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 21 13:18:49.591773 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:49.591257    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf" podUID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 21 13:18:50.591674 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:50.591622    2563 patch_prober.go:28] interesting pod/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 21 13:18:50.592171 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:50.591698    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf" podUID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 21 13:18:51.590969 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:51.590924    2563 patch_prober.go:28] interesting pod/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 21 13:18:51.591181 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:51.590991    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf" podUID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 21 13:18:52.594609 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:52.594578    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:52.594996 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:52.594934    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:18:52.595658 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:18:52.595636    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:19:07.240662 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:07.240619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:11.757031 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:11.756988    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:11.878698 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:11.878664    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"]
Jul 21 13:19:11.878964 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:11.878903    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf" podUID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerName="istio-proxy" containerID="cri-o://e56ada89169c401faa6d37605ee718e7b0e98e4ee283bbb3261762eb3cc9cafc" gracePeriod=30
Jul 21 13:19:11.925218 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:11.925164    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:12.144990 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.144959    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:12.385930 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.385876    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:12.710493 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.710459    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"]
Jul 21 13:19:12.710810 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.710796    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68aaf89b-50f4-497c-ab1f-2144c757fc70" containerName="istio-proxy"
Jul 21 13:19:12.710885 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.710812    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="68aaf89b-50f4-497c-ab1f-2144c757fc70" containerName="istio-proxy"
Jul 21 13:19:12.710885 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.710827    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68aaf89b-50f4-497c-ab1f-2144c757fc70" containerName="istio-proxy"
Jul 21 13:19:12.710885 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.710832    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="68aaf89b-50f4-497c-ab1f-2144c757fc70" containerName="istio-proxy"
Jul 21 13:19:12.711035 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.710908    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="68aaf89b-50f4-497c-ab1f-2144c757fc70" containerName="istio-proxy"
Jul 21 13:19:12.711035 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.710916    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="68aaf89b-50f4-497c-ab1f-2144c757fc70" containerName="istio-proxy"
Jul 21 13:19:12.714788 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.714770    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.717943 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.717923    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a6824535-openshift-default-dockercfg-dxnmp\""
Jul 21 13:19:12.729704 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.729676    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"]
Jul 21 13:19:12.755508 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.755470    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:12.789355 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.789282    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-istio-data\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.789721 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.789365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6683353-c795-4908-b557-ad6d89ad4877-istio-token\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.789721 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.789386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6683353-c795-4908-b557-ad6d89ad4877-istiod-ca-cert\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.789721 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.789407    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-workload-socket\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.789721 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.789422    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-workload-certs\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.789721 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.789499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-credential-socket\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.789721 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.789574    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6683353-c795-4908-b557-ad6d89ad4877-istio-podinfo\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.789721 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.789593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvbh6\" (UniqueName: \"kubernetes.io/projected/d6683353-c795-4908-b557-ad6d89ad4877-kube-api-access-hvbh6\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.789721 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.789667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-istio-envoy\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.854441 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.854401    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:12.891173 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.891117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6683353-c795-4908-b557-ad6d89ad4877-istio-podinfo\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.891392 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.891182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvbh6\" (UniqueName: \"kubernetes.io/projected/d6683353-c795-4908-b557-ad6d89ad4877-kube-api-access-hvbh6\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.891392 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.891258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-istio-envoy\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.891392 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.891343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-istio-data\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.891392 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.891376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6683353-c795-4908-b557-ad6d89ad4877-istio-token\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.891614 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.891407    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6683353-c795-4908-b557-ad6d89ad4877-istiod-ca-cert\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.891614 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.891441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-workload-socket\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.891614 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.891466    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-workload-certs\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.891614 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.891495    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-credential-socket\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.892024 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.891984    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-credential-socket\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.893066 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.892583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-workload-socket\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.893066 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.892712    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6683353-c795-4908-b557-ad6d89ad4877-istiod-ca-cert\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.893066 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.893024    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-workload-certs\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.893066 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.893058    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-istio-data\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.894815 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.894769    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6683353-c795-4908-b557-ad6d89ad4877-istio-podinfo\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.895032 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.894995    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-istio-envoy\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.901334 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.901283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvbh6\" (UniqueName: \"kubernetes.io/projected/d6683353-c795-4908-b557-ad6d89ad4877-kube-api-access-hvbh6\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:12.901451 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:12.901364    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6683353-c795-4908-b557-ad6d89ad4877-istio-token\") pod \"e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") " pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:13.026414 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:13.026321    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:13.193453 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:13.193426    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"]
Jul 21 13:19:13.195249 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:19:13.195222    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6683353_c795_4908_b557_ad6d89ad4877.slice/crio-c5b444b3ebaa7a10547a341f410aab99522d649bddff9e76f7669ac86a207dd1 WatchSource:0}: Error finding container c5b444b3ebaa7a10547a341f410aab99522d649bddff9e76f7669ac86a207dd1: Status 404 returned error can't find the container with id c5b444b3ebaa7a10547a341f410aab99522d649bddff9e76f7669ac86a207dd1
Jul 21 13:19:13.197850 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:13.197816    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:19:13.197932 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:13.197887    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:19:13.197932 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:13.197916    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:19:13.218394 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:13.218367    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc" event={"ID":"d6683353-c795-4908-b557-ad6d89ad4877","Type":"ContainerStarted","Data":"c5b444b3ebaa7a10547a341f410aab99522d649bddff9e76f7669ac86a207dd1"}
Jul 21 13:19:14.225020 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:14.224985    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc" event={"ID":"d6683353-c795-4908-b557-ad6d89ad4877","Type":"ContainerStarted","Data":"f786c2f4e8307169c44ce98bd0260288157ad9e488e8ebd5b977fd842e6aa6cb"}
Jul 21 13:19:14.250131 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:14.250077    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc" podStartSLOduration=2.250054841 podStartE2EDuration="2.250054841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:19:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:19:14.248659286 +0000 UTC m=+1686.774162455" watchObservedRunningTime="2026-07-21 13:19:14.250054841 +0000 UTC m=+1686.775557997"
Jul 21 13:19:15.027163 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:15.027117    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:15.032362 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:15.032331    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:15.229859 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:15.229820    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:15.230952 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:15.230931    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:15.612558 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:15.612521    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:15.695006 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:15.694977    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:15.791867 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:15.791837    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:16.002237 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:16.002149    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:16.381689 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:16.381658    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:16.492148 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:16.492116    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:17.144910 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.144845    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:19:17.235019 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.234988    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xw79\" (UniqueName: \"kubernetes.io/projected/c3da855f-ddea-43d5-a63f-b1632c5cd45a-kube-api-access-9xw79\") pod \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") "
Jul 21 13:19:17.235019 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.235025    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-token\") pod \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") "
Jul 21 13:19:17.235278 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.235080    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-envoy\") pod \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") "
Jul 21 13:19:17.235278 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.235103    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-credential-socket\") pod \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") "
Jul 21 13:19:17.235278 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.235161    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-podinfo\") pod \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") "
Jul 21 13:19:17.235278 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.235233    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istiod-ca-cert\") pod \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") "
Jul 21 13:19:17.235278 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.235266    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-workload-socket\") pod \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") "
Jul 21 13:19:17.235578 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.235340    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-data\") pod \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") "
Jul 21 13:19:17.235578 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.235384    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-workload-certs\") pod \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\" (UID: \"c3da855f-ddea-43d5-a63f-b1632c5cd45a\") "
Jul 21 13:19:17.235578 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.235451    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c3da855f-ddea-43d5-a63f-b1632c5cd45a" (UID: "c3da855f-ddea-43d5-a63f-b1632c5cd45a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:19:17.235805 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.235783    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-credential-socket\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:19:17.236222 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.236190    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c3da855f-ddea-43d5-a63f-b1632c5cd45a" (UID: "c3da855f-ddea-43d5-a63f-b1632c5cd45a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:19:17.236379 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.236353    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c3da855f-ddea-43d5-a63f-b1632c5cd45a" (UID: "c3da855f-ddea-43d5-a63f-b1632c5cd45a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:19:17.236524 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.236384    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c3da855f-ddea-43d5-a63f-b1632c5cd45a" (UID: "c3da855f-ddea-43d5-a63f-b1632c5cd45a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:19:17.236647 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.236442    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-data" (OuterVolumeSpecName: "istio-data") pod "c3da855f-ddea-43d5-a63f-b1632c5cd45a" (UID: "c3da855f-ddea-43d5-a63f-b1632c5cd45a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:19:17.237742 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.237706    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c3da855f-ddea-43d5-a63f-b1632c5cd45a" (UID: "c3da855f-ddea-43d5-a63f-b1632c5cd45a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:19:17.237742 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.237716    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-token" (OuterVolumeSpecName: "istio-token") pod "c3da855f-ddea-43d5-a63f-b1632c5cd45a" (UID: "c3da855f-ddea-43d5-a63f-b1632c5cd45a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:19:17.238031 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.238010    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3da855f-ddea-43d5-a63f-b1632c5cd45a-kube-api-access-9xw79" (OuterVolumeSpecName: "kube-api-access-9xw79") pod "c3da855f-ddea-43d5-a63f-b1632c5cd45a" (UID: "c3da855f-ddea-43d5-a63f-b1632c5cd45a"). InnerVolumeSpecName "kube-api-access-9xw79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:19:17.238098 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.238076    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c3da855f-ddea-43d5-a63f-b1632c5cd45a" (UID: "c3da855f-ddea-43d5-a63f-b1632c5cd45a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:19:17.242184 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.242157    2563 generic.go:356] "Generic (PLEG): container finished" podID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerID="e56ada89169c401faa6d37605ee718e7b0e98e4ee283bbb3261762eb3cc9cafc" exitCode=0
Jul 21 13:19:17.242316 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.242212    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"
Jul 21 13:19:17.242316 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.242244    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf" event={"ID":"c3da855f-ddea-43d5-a63f-b1632c5cd45a","Type":"ContainerDied","Data":"e56ada89169c401faa6d37605ee718e7b0e98e4ee283bbb3261762eb3cc9cafc"}
Jul 21 13:19:17.242428 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.242346    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf" event={"ID":"c3da855f-ddea-43d5-a63f-b1632c5cd45a","Type":"ContainerDied","Data":"d83092de4c7db98b23a50b5c3786923ff86f55484d8ca14040b4db9c1b2a7b6c"}
Jul 21 13:19:17.242428 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.242380    2563 scope.go:117] "RemoveContainer" containerID="e56ada89169c401faa6d37605ee718e7b0e98e4ee283bbb3261762eb3cc9cafc"
Jul 21 13:19:17.256793 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.256776    2563 scope.go:117] "RemoveContainer" containerID="da26be29de20303ed527f2a0d490d44c098c11a799ab1311e6369e58151ae8d5"
Jul 21 13:19:17.264473 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.264453    2563 scope.go:117] "RemoveContainer" containerID="e56ada89169c401faa6d37605ee718e7b0e98e4ee283bbb3261762eb3cc9cafc"
Jul 21 13:19:17.264756 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:19:17.264736    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e56ada89169c401faa6d37605ee718e7b0e98e4ee283bbb3261762eb3cc9cafc\": container with ID starting with e56ada89169c401faa6d37605ee718e7b0e98e4ee283bbb3261762eb3cc9cafc not found: ID does not exist" containerID="e56ada89169c401faa6d37605ee718e7b0e98e4ee283bbb3261762eb3cc9cafc"
Jul 21 13:19:17.264809 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.264765    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e56ada89169c401faa6d37605ee718e7b0e98e4ee283bbb3261762eb3cc9cafc"} err="failed to get container status \"e56ada89169c401faa6d37605ee718e7b0e98e4ee283bbb3261762eb3cc9cafc\": rpc error: code = NotFound desc = could not find container \"e56ada89169c401faa6d37605ee718e7b0e98e4ee283bbb3261762eb3cc9cafc\": container with ID starting with e56ada89169c401faa6d37605ee718e7b0e98e4ee283bbb3261762eb3cc9cafc not found: ID does not exist"
Jul 21 13:19:17.264809 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.264784    2563 scope.go:117] "RemoveContainer" containerID="da26be29de20303ed527f2a0d490d44c098c11a799ab1311e6369e58151ae8d5"
Jul 21 13:19:17.265048 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:19:17.265028    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da26be29de20303ed527f2a0d490d44c098c11a799ab1311e6369e58151ae8d5\": container with ID starting with da26be29de20303ed527f2a0d490d44c098c11a799ab1311e6369e58151ae8d5 not found: ID does not exist" containerID="da26be29de20303ed527f2a0d490d44c098c11a799ab1311e6369e58151ae8d5"
Jul 21 13:19:17.265084 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.265055    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da26be29de20303ed527f2a0d490d44c098c11a799ab1311e6369e58151ae8d5"} err="failed to get container status \"da26be29de20303ed527f2a0d490d44c098c11a799ab1311e6369e58151ae8d5\": rpc error: code = NotFound desc = could not find container \"da26be29de20303ed527f2a0d490d44c098c11a799ab1311e6369e58151ae8d5\": container with ID starting with da26be29de20303ed527f2a0d490d44c098c11a799ab1311e6369e58151ae8d5 not found: ID does not exist"
Jul 21 13:19:17.277075 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.277031    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"]
Jul 21 13:19:17.287759 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.287726    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-8bfb8415-gw-openshift-default-55bcfb787-xvntf"]
Jul 21 13:19:17.337051 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.337014    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-data\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:19:17.337051 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.337045    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-workload-certs\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:19:17.337051 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.337059    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xw79\" (UniqueName: \"kubernetes.io/projected/c3da855f-ddea-43d5-a63f-b1632c5cd45a-kube-api-access-9xw79\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:19:17.337313 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.337072    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-token\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:19:17.337313 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.337084    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-envoy\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:19:17.337313 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.337095    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istio-podinfo\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:19:17.337313 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.337104    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3da855f-ddea-43d5-a63f-b1632c5cd45a-istiod-ca-cert\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:19:17.337313 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.337113    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3da855f-ddea-43d5-a63f-b1632c5cd45a-workload-socket\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:19:17.995511 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:17.995482    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" path="/var/lib/kubelet/pods/c3da855f-ddea-43d5-a63f-b1632c5cd45a/volumes"
Jul 21 13:19:29.310122 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:29.310042    2563 generic.go:356] "Generic (PLEG): container finished" podID="d6683353-c795-4908-b557-ad6d89ad4877" containerID="f786c2f4e8307169c44ce98bd0260288157ad9e488e8ebd5b977fd842e6aa6cb" exitCode=0
Jul 21 13:19:29.310122 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:29.310116    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc" event={"ID":"d6683353-c795-4908-b557-ad6d89ad4877","Type":"ContainerDied","Data":"f786c2f4e8307169c44ce98bd0260288157ad9e488e8ebd5b977fd842e6aa6cb"}
Jul 21 13:19:29.310576 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:29.310523    2563 scope.go:117] "RemoveContainer" containerID="f786c2f4e8307169c44ce98bd0260288157ad9e488e8ebd5b977fd842e6aa6cb"
Jul 21 13:19:29.311598 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:29.311570    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:19:29.311714 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:29.311648    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:19:29.311714 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:29.311684    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 21 13:19:30.230247 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:30.230202    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:30.316151 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:30.316115    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc" event={"ID":"d6683353-c795-4908-b557-ad6d89ad4877","Type":"ContainerStarted","Data":"ce67a2e213ba251816b83c4429ae4dd76ee4c9fc883df635c36104dc6da53fe3"}
Jul 21 13:19:31.027356 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:31.027310    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:31.028467 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:31.028437    2563 patch_prober.go:28] interesting pod/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 21 13:19:31.028604 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:31.028506    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc" podUID="d6683353-c795-4908-b557-ad6d89ad4877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 21 13:19:32.027490 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:32.027451    2563 patch_prober.go:28] interesting pod/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 21 13:19:32.027890 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:32.027513    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc" podUID="d6683353-c795-4908-b557-ad6d89ad4877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 21 13:19:33.027710 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:33.027675    2563 patch_prober.go:28] interesting pod/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 21 13:19:33.028115 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:33.027733    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc" podUID="d6683353-c795-4908-b557-ad6d89ad4877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 21 13:19:34.026716 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:34.026674    2563 patch_prober.go:28] interesting pod/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 21 13:19:34.026896 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:34.026736    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc" podUID="d6683353-c795-4908-b557-ad6d89ad4877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 21 13:19:35.030691 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:35.030660    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:35.031077 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:35.030965    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:35.031657 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:35.031641    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:19:52.243495 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:52.243463    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:56.742675 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:56.742633    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:56.877832 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:56.877788    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:58.231539 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:58.231503    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"]
Jul 21 13:19:58.231950 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:58.231731    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc" podUID="d6683353-c795-4908-b557-ad6d89ad4877" containerName="istio-proxy" containerID="cri-o://ce67a2e213ba251816b83c4429ae4dd76ee4c9fc883df635c36104dc6da53fe3" gracePeriod=30
Jul 21 13:19:58.265496 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:58.265455    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:58.933052 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:58.933009    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:59.029082 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:59.029046    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:59.256479 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:59.256388    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:19:59.426566 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:19:59.426529    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:03.462064 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.462034    2563 generic.go:356] "Generic (PLEG): container finished" podID="d6683353-c795-4908-b557-ad6d89ad4877" containerID="ce67a2e213ba251816b83c4429ae4dd76ee4c9fc883df635c36104dc6da53fe3" exitCode=0
Jul 21 13:20:03.462537 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.462073    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc" event={"ID":"d6683353-c795-4908-b557-ad6d89ad4877","Type":"ContainerDied","Data":"ce67a2e213ba251816b83c4429ae4dd76ee4c9fc883df635c36104dc6da53fe3"}
Jul 21 13:20:03.462537 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.462122    2563 scope.go:117] "RemoveContainer" containerID="f786c2f4e8307169c44ce98bd0260288157ad9e488e8ebd5b977fd842e6aa6cb"
Jul 21 13:20:03.489186 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.489165    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:20:03.568856 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.568754    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-credential-socket\") pod \"d6683353-c795-4908-b557-ad6d89ad4877\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") "
Jul 21 13:20:03.568856 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.568825    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-workload-certs\") pod \"d6683353-c795-4908-b557-ad6d89ad4877\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") "
Jul 21 13:20:03.569098 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.568862    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6683353-c795-4908-b557-ad6d89ad4877-istio-token\") pod \"d6683353-c795-4908-b557-ad6d89ad4877\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") "
Jul 21 13:20:03.569098 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.568892    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-istio-envoy\") pod \"d6683353-c795-4908-b557-ad6d89ad4877\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") "
Jul 21 13:20:03.569098 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.568929    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvbh6\" (UniqueName: \"kubernetes.io/projected/d6683353-c795-4908-b557-ad6d89ad4877-kube-api-access-hvbh6\") pod \"d6683353-c795-4908-b557-ad6d89ad4877\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") "
Jul 21 13:20:03.569098 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.568993    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6683353-c795-4908-b557-ad6d89ad4877-istio-podinfo\") pod \"d6683353-c795-4908-b557-ad6d89ad4877\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") "
Jul 21 13:20:03.569098 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.569018    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-workload-socket\") pod \"d6683353-c795-4908-b557-ad6d89ad4877\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") "
Jul 21 13:20:03.569098 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.569057    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d6683353-c795-4908-b557-ad6d89ad4877" (UID: "d6683353-c795-4908-b557-ad6d89ad4877"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:20:03.569457 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.569161    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d6683353-c795-4908-b557-ad6d89ad4877" (UID: "d6683353-c795-4908-b557-ad6d89ad4877"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:20:03.569457 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.569370    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d6683353-c795-4908-b557-ad6d89ad4877" (UID: "d6683353-c795-4908-b557-ad6d89ad4877"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:20:03.569457 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.569433    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-istio-data\") pod \"d6683353-c795-4908-b557-ad6d89ad4877\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") "
Jul 21 13:20:03.569609 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.569465    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6683353-c795-4908-b557-ad6d89ad4877-istiod-ca-cert\") pod \"d6683353-c795-4908-b557-ad6d89ad4877\" (UID: \"d6683353-c795-4908-b557-ad6d89ad4877\") "
Jul 21 13:20:03.569780 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.569760    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-credential-socket\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.569863 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.569797    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-workload-certs\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.569863 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.569809    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6683353-c795-4908-b557-ad6d89ad4877-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d6683353-c795-4908-b557-ad6d89ad4877" (UID: "d6683353-c795-4908-b557-ad6d89ad4877"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:20:03.569863 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.569829    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-workload-socket\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.570107 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.570089    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-istio-data" (OuterVolumeSpecName: "istio-data") pod "d6683353-c795-4908-b557-ad6d89ad4877" (UID: "d6683353-c795-4908-b557-ad6d89ad4877"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:20:03.571968 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.571946    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d6683353-c795-4908-b557-ad6d89ad4877" (UID: "d6683353-c795-4908-b557-ad6d89ad4877"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:20:03.572042 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.571976    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6683353-c795-4908-b557-ad6d89ad4877-istio-token" (OuterVolumeSpecName: "istio-token") pod "d6683353-c795-4908-b557-ad6d89ad4877" (UID: "d6683353-c795-4908-b557-ad6d89ad4877"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:20:03.572042 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.571977    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6683353-c795-4908-b557-ad6d89ad4877-kube-api-access-hvbh6" (OuterVolumeSpecName: "kube-api-access-hvbh6") pod "d6683353-c795-4908-b557-ad6d89ad4877" (UID: "d6683353-c795-4908-b557-ad6d89ad4877"). InnerVolumeSpecName "kube-api-access-hvbh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:20:03.572042 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.572000    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6683353-c795-4908-b557-ad6d89ad4877-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d6683353-c795-4908-b557-ad6d89ad4877" (UID: "d6683353-c795-4908-b557-ad6d89ad4877"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:20:03.670318 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.670243    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6683353-c795-4908-b557-ad6d89ad4877-istio-token\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.670318 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.670280    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-istio-envoy\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.670318 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.670323    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvbh6\" (UniqueName: \"kubernetes.io/projected/d6683353-c795-4908-b557-ad6d89ad4877-kube-api-access-hvbh6\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.670566 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.670337    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6683353-c795-4908-b557-ad6d89ad4877-istio-podinfo\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.670566 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.670350    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6683353-c795-4908-b557-ad6d89ad4877-istio-data\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.670566 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:03.670360    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6683353-c795-4908-b557-ad6d89ad4877-istiod-ca-cert\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:20:04.468232 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:04.468198    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc" event={"ID":"d6683353-c795-4908-b557-ad6d89ad4877","Type":"ContainerDied","Data":"c5b444b3ebaa7a10547a341f410aab99522d649bddff9e76f7669ac86a207dd1"}
Jul 21 13:20:04.468232 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:04.468212    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"
Jul 21 13:20:04.468759 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:04.468316    2563 scope.go:117] "RemoveContainer" containerID="ce67a2e213ba251816b83c4429ae4dd76ee4c9fc883df635c36104dc6da53fe3"
Jul 21 13:20:04.487823 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:04.487790    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"]
Jul 21 13:20:04.492452 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:04.492420    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a6824535-openshift-default-dcc747bc9-vbrsc"]
Jul 21 13:20:05.003642 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:05.003613    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:05.133464 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:05.133434    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:05.234160 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:05.234112    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:05.380593 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:05.380563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:05.738093 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:05.738014    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:05.839123 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:05.839090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:05.996219 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:05.996123    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6683353-c795-4908-b557-ad6d89ad4877" path="/var/lib/kubelet/pods/d6683353-c795-4908-b557-ad6d89ad4877/volumes"
Jul 21 13:20:06.875039 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:06.875008    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:06.959119 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:06.959086    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:07.848231 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:07.848196    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:08.297854 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:08.297818    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:30.024743 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.024674    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv"]
Jul 21 13:20:30.025242 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.025054    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerName="istio-proxy"
Jul 21 13:20:30.025242 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.025067    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerName="istio-proxy"
Jul 21 13:20:30.025242 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.025081    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6683353-c795-4908-b557-ad6d89ad4877" containerName="istio-proxy"
Jul 21 13:20:30.025242 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.025086    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6683353-c795-4908-b557-ad6d89ad4877" containerName="istio-proxy"
Jul 21 13:20:30.025242 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.025096    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6683353-c795-4908-b557-ad6d89ad4877" containerName="istio-proxy"
Jul 21 13:20:30.025242 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.025101    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6683353-c795-4908-b557-ad6d89ad4877" containerName="istio-proxy"
Jul 21 13:20:30.025242 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.025110    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerName="istio-proxy"
Jul 21 13:20:30.025242 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.025120    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerName="istio-proxy"
Jul 21 13:20:30.025242 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.025170    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerName="istio-proxy"
Jul 21 13:20:30.025242 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.025178    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6683353-c795-4908-b557-ad6d89ad4877" containerName="istio-proxy"
Jul 21 13:20:30.025242 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.025184    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3da855f-ddea-43d5-a63f-b1632c5cd45a" containerName="istio-proxy"
Jul 21 13:20:30.029430 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.029412    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv"
Jul 21 13:20:30.032860 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.032836    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 13:20:30.032860 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.032865    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qhp7l\""
Jul 21 13:20:30.038031 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.038000    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv"]
Jul 21 13:20:30.107569 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.107540    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dc49e05-4e68-4b2c-88e6-45de81a2370f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-222aaf00-24cbv\" (UID: \"0dc49e05-4e68-4b2c-88e6-45de81a2370f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv"
Jul 21 13:20:30.208917 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.208865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dc49e05-4e68-4b2c-88e6-45de81a2370f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-222aaf00-24cbv\" (UID: \"0dc49e05-4e68-4b2c-88e6-45de81a2370f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv"
Jul 21 13:20:30.209547 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.209525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dc49e05-4e68-4b2c-88e6-45de81a2370f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-222aaf00-24cbv\" (UID: \"0dc49e05-4e68-4b2c-88e6-45de81a2370f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv"
Jul 21 13:20:30.342428 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.342317    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv"
Jul 21 13:20:30.482961 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.482937    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv"]
Jul 21 13:20:30.485144 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:20:30.485105    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dc49e05_4e68_4b2c_88e6_45de81a2370f.slice/crio-dffc2cbc39532b6d690a48db7353d7328e719fe6000bb1000bea8f5e3ed23437 WatchSource:0}: Error finding container dffc2cbc39532b6d690a48db7353d7328e719fe6000bb1000bea8f5e3ed23437: Status 404 returned error can't find the container with id dffc2cbc39532b6d690a48db7353d7328e719fe6000bb1000bea8f5e3ed23437
Jul 21 13:20:30.486845 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.486829    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 13:20:30.578829 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:30.578792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv" event={"ID":"0dc49e05-4e68-4b2c-88e6-45de81a2370f","Type":"ContainerStarted","Data":"dffc2cbc39532b6d690a48db7353d7328e719fe6000bb1000bea8f5e3ed23437"}
Jul 21 13:20:33.594503 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:33.594417    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv" event={"ID":"0dc49e05-4e68-4b2c-88e6-45de81a2370f","Type":"ContainerStarted","Data":"d572d7392864b510c626406f5029f2bfea9f4a66e5d52ebc9df77738c44d62fc"}
Jul 21 13:20:33.612539 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:33.612479    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv" podStartSLOduration=1.219507795 podStartE2EDuration="3.612463739s" totalImagesPullingTime="2.392955944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:20:33.611427288 +0000 UTC m=+1766.136930444" watchObservedRunningTime="2026-07-21 13:20:33.612463739 +0000 UTC m=+1766.137966895"
Jul 21 13:20:34.601870 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:34.601832    2563 generic.go:356] "Generic (PLEG): container finished" podID="0dc49e05-4e68-4b2c-88e6-45de81a2370f" containerID="d572d7392864b510c626406f5029f2bfea9f4a66e5d52ebc9df77738c44d62fc" exitCode=0
Jul 21 13:20:34.602330 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:34.601916    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv" event={"ID":"0dc49e05-4e68-4b2c-88e6-45de81a2370f","Type":"ContainerDied","Data":"d572d7392864b510c626406f5029f2bfea9f4a66e5d52ebc9df77738c44d62fc"}
Jul 21 13:20:35.740448 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:35.740424    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv"
Jul 21 13:20:35.863524 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:35.863427    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dc49e05-4e68-4b2c-88e6-45de81a2370f-maas-api-service-ca\") pod \"0dc49e05-4e68-4b2c-88e6-45de81a2370f\" (UID: \"0dc49e05-4e68-4b2c-88e6-45de81a2370f\") "
Jul 21 13:20:35.863811 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:35.863785    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dc49e05-4e68-4b2c-88e6-45de81a2370f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0dc49e05-4e68-4b2c-88e6-45de81a2370f" (UID: "0dc49e05-4e68-4b2c-88e6-45de81a2370f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:20:35.965004 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:35.964957    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dc49e05-4e68-4b2c-88e6-45de81a2370f-maas-api-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:20:36.611160 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:36.611136    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv"
Jul 21 13:20:36.611160 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:36.611150    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv" event={"ID":"0dc49e05-4e68-4b2c-88e6-45de81a2370f","Type":"ContainerDied","Data":"dffc2cbc39532b6d690a48db7353d7328e719fe6000bb1000bea8f5e3ed23437"}
Jul 21 13:20:36.611436 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:36.611194    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dffc2cbc39532b6d690a48db7353d7328e719fe6000bb1000bea8f5e3ed23437"
Jul 21 13:20:40.578961 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:40.578926    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:45.107102 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:45.107070    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:45.114632 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:45.114597    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv"]
Jul 21 13:20:45.117028 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:45.117003    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-222aaf00-24cbv"]
Jul 21 13:20:45.370436 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:45.370323    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:45.563776 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:45.563743    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:45.996033 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:45.995996    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dc49e05-4e68-4b2c-88e6-45de81a2370f" path="/var/lib/kubelet/pods/0dc49e05-4e68-4b2c-88e6-45de81a2370f/volumes"
Jul 21 13:20:46.972856 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:46.972822    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:47.075100 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:47.075053    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:47.986801 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:47.986769    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:53.812926 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:53.812880    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:53.931959 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:53.931923    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:54.037868 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:54.037836    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:54.129982 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:54.129947    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:20:55.194800 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:20:55.194763    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:21:08.044936 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:21:08.044904    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:21:08.050011 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:21:08.049989    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:21:20.690132 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:21:20.690092    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:21:27.653826 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:21:27.653798    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:21:27.753677 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:21:27.753640    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:21:27.976174 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:21:27.976087    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:21:43.890265 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:21:43.890232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:21:49.872126 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:21:49.872088    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:21:49.936807 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:21:49.936767    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:21:50.049955 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:21:50.049906    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:21:50.139327 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:21:50.139189    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:00.654764 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:00.654710    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:01.546101 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:01.546064    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:08.000122 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:08.000090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:08.100334 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:08.100280    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:08.199914 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:08.199857    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:08.301471 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:08.301377    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:08.405701 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:08.405671    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:09.957049 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:09.957013    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:10.043042 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:10.042998    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:11.218072 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:11.218035    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:17.656317 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:17.656261    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:17.754063 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:17.754026    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:18.404816 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:18.404785    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:24.432494 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:24.432459    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:24.494726 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:24.494688    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:24.721365 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:24.721257    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:25.052507 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:25.052413    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:26.188050 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:26.188016    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:28.585565 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.585472    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:28.667016 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.666986    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:28.673054 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.673024    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"]
Jul 21 13:22:28.673501 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.673486    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dc49e05-4e68-4b2c-88e6-45de81a2370f" containerName="revoke-keys"
Jul 21 13:22:28.673549 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.673505    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dc49e05-4e68-4b2c-88e6-45de81a2370f" containerName="revoke-keys"
Jul 21 13:22:28.673595 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.673583    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dc49e05-4e68-4b2c-88e6-45de81a2370f" containerName="revoke-keys"
Jul 21 13:22:28.673630 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.673601    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6683353-c795-4908-b557-ad6d89ad4877" containerName="istio-proxy"
Jul 21 13:22:28.676901 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.676879    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:28.679816 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.679792    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 13:22:28.680109 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.680088    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pkltq\""
Jul 21 13:22:28.680958 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.680934    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 13:22:28.681056 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.680964    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ea9fc8fd-serving-cert\""
Jul 21 13:22:28.692578 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.692548    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"]
Jul 21 13:22:28.749661 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.749626    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:28.781004 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.780958    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jqlb\" (UniqueName: \"kubernetes.io/projected/cdbc746e-708d-4871-8a3a-a9ecec02b661-kube-api-access-4jqlb\") pod \"maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b\" (UID: \"cdbc746e-708d-4871-8a3a-a9ecec02b661\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:28.781171 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.781058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbc746e-708d-4871-8a3a-a9ecec02b661-maas-api-tls\") pod \"maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b\" (UID: \"cdbc746e-708d-4871-8a3a-a9ecec02b661\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:28.781171 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.781086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbc746e-708d-4871-8a3a-a9ecec02b661-openshift-service-ca\") pod \"maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b\" (UID: \"cdbc746e-708d-4871-8a3a-a9ecec02b661\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:28.882671 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.882629    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jqlb\" (UniqueName: \"kubernetes.io/projected/cdbc746e-708d-4871-8a3a-a9ecec02b661-kube-api-access-4jqlb\") pod \"maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b\" (UID: \"cdbc746e-708d-4871-8a3a-a9ecec02b661\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:28.882883 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.882709    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbc746e-708d-4871-8a3a-a9ecec02b661-maas-api-tls\") pod \"maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b\" (UID: \"cdbc746e-708d-4871-8a3a-a9ecec02b661\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:28.882883 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.882734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbc746e-708d-4871-8a3a-a9ecec02b661-openshift-service-ca\") pod \"maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b\" (UID: \"cdbc746e-708d-4871-8a3a-a9ecec02b661\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:28.882883 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:22:28.882861    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-serving-cert: secret "maas-api-e2e-mt-ea9fc8fd-serving-cert" not found
Jul 21 13:22:28.883057 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:22:28.882935    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdbc746e-708d-4871-8a3a-a9ecec02b661-maas-api-tls podName:cdbc746e-708d-4871-8a3a-a9ecec02b661 nodeName:}" failed. No retries permitted until 2026-07-21 13:22:29.382916407 +0000 UTC m=+1881.908419549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cdbc746e-708d-4871-8a3a-a9ecec02b661-maas-api-tls") pod "maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b" (UID: "cdbc746e-708d-4871-8a3a-a9ecec02b661") : secret "maas-api-e2e-mt-ea9fc8fd-serving-cert" not found
Jul 21 13:22:28.883533 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.883510    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbc746e-708d-4871-8a3a-a9ecec02b661-openshift-service-ca\") pod \"maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b\" (UID: \"cdbc746e-708d-4871-8a3a-a9ecec02b661\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:28.892565 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.892534    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jqlb\" (UniqueName: \"kubernetes.io/projected/cdbc746e-708d-4871-8a3a-a9ecec02b661-kube-api-access-4jqlb\") pod \"maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b\" (UID: \"cdbc746e-708d-4871-8a3a-a9ecec02b661\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:28.977009 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:28.976977    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:29.305811 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:29.305728    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:29.387159 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:29.387121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbc746e-708d-4871-8a3a-a9ecec02b661-maas-api-tls\") pod \"maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b\" (UID: \"cdbc746e-708d-4871-8a3a-a9ecec02b661\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:29.389853 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:29.389829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbc746e-708d-4871-8a3a-a9ecec02b661-maas-api-tls\") pod \"maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b\" (UID: \"cdbc746e-708d-4871-8a3a-a9ecec02b661\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:29.403876 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:29.403844    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:29.589051 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:29.588951    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:29.730822 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:29.730797    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"]
Jul 21 13:22:29.732894 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:22:29.732866    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdbc746e_708d_4871_8a3a_a9ecec02b661.slice/crio-c1eaa551309effeffb8939cd0ed5b7263333397976c68a589578aa1607f78991 WatchSource:0}: Error finding container c1eaa551309effeffb8939cd0ed5b7263333397976c68a589578aa1607f78991: Status 404 returned error can't find the container with id c1eaa551309effeffb8939cd0ed5b7263333397976c68a589578aa1607f78991
Jul 21 13:22:30.105172 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:30.105143    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b" event={"ID":"cdbc746e-708d-4871-8a3a-a9ecec02b661","Type":"ContainerStarted","Data":"c1eaa551309effeffb8939cd0ed5b7263333397976c68a589578aa1607f78991"}
Jul 21 13:22:32.118168 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:32.118135    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b" event={"ID":"cdbc746e-708d-4871-8a3a-a9ecec02b661","Type":"ContainerStarted","Data":"603cb360237814c963bd48d3312d8b2119d25ec4338fe16d2de3d9a796c8d66c"}
Jul 21 13:22:32.118664 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:32.118264    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:32.136729 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:32.136669    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b" podStartSLOduration=2.392906484 podStartE2EDuration="4.136643085s" totalImagesPullingTime="1.743736601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:22:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:22:32.135615037 +0000 UTC m=+1884.661118192" watchObservedRunningTime="2026-07-21 13:22:32.136643085 +0000 UTC m=+1884.662146240"
Jul 21 13:22:33.036702 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:33.036658    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:33.823657 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:33.823626    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:33.922273 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:33.922237    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:34.319572 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:34.319535    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:38.129508 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:38.129478    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:40.583540 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:40.583501    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5"]
Jul 21 13:22:40.586644 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:40.586626    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5"
Jul 21 13:22:40.589015 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:40.588999    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qhp7l\""
Jul 21 13:22:40.594980 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:40.594943    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5"]
Jul 21 13:22:40.686989 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:40.686939    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99312641-bc52-420d-b730-9161c95dff0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5\" (UID: \"99312641-bc52-420d-b730-9161c95dff0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5"
Jul 21 13:22:40.788146 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:40.788108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99312641-bc52-420d-b730-9161c95dff0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5\" (UID: \"99312641-bc52-420d-b730-9161c95dff0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5"
Jul 21 13:22:40.788797 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:40.788775    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99312641-bc52-420d-b730-9161c95dff0a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5\" (UID: \"99312641-bc52-420d-b730-9161c95dff0a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5"
Jul 21 13:22:40.856219 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:40.856186    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:40.897632 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:40.897597    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5"
Jul 21 13:22:40.962951 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:40.962703    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:41.238325 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:41.238262    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5"]
Jul 21 13:22:41.241013 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:22:41.240986    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99312641_bc52_420d_b730_9161c95dff0a.slice/crio-c0f635cb2fc3ea0ae61e2f0125780ecd748fe7ac2a09958b3623ec49cd8ee723 WatchSource:0}: Error finding container c0f635cb2fc3ea0ae61e2f0125780ecd748fe7ac2a09958b3623ec49cd8ee723: Status 404 returned error can't find the container with id c0f635cb2fc3ea0ae61e2f0125780ecd748fe7ac2a09958b3623ec49cd8ee723
Jul 21 13:22:42.165041 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:42.165008    2563 generic.go:356] "Generic (PLEG): container finished" podID="99312641-bc52-420d-b730-9161c95dff0a" containerID="0f21548de73b805f348bc3c50ac7d63400e64453c024830d7bdadd9d620784ea" exitCode=0
Jul 21 13:22:42.165452 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:42.165099    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5" event={"ID":"99312641-bc52-420d-b730-9161c95dff0a","Type":"ContainerDied","Data":"0f21548de73b805f348bc3c50ac7d63400e64453c024830d7bdadd9d620784ea"}
Jul 21 13:22:42.165452 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:42.165167    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5" event={"ID":"99312641-bc52-420d-b730-9161c95dff0a","Type":"ContainerStarted","Data":"c0f635cb2fc3ea0ae61e2f0125780ecd748fe7ac2a09958b3623ec49cd8ee723"}
Jul 21 13:22:43.291496 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:43.291469    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5"
Jul 21 13:22:43.310680 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:43.310648    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99312641-bc52-420d-b730-9161c95dff0a-maas-api-service-ca\") pod \"99312641-bc52-420d-b730-9161c95dff0a\" (UID: \"99312641-bc52-420d-b730-9161c95dff0a\") "
Jul 21 13:22:43.311049 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:43.311020    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99312641-bc52-420d-b730-9161c95dff0a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "99312641-bc52-420d-b730-9161c95dff0a" (UID: "99312641-bc52-420d-b730-9161c95dff0a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:22:43.411887 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:43.411851    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99312641-bc52-420d-b730-9161c95dff0a-maas-api-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:22:44.174594 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:44.174560    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5"
Jul 21 13:22:44.174771 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:44.174569    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5" event={"ID":"99312641-bc52-420d-b730-9161c95dff0a","Type":"ContainerDied","Data":"c0f635cb2fc3ea0ae61e2f0125780ecd748fe7ac2a09958b3623ec49cd8ee723"}
Jul 21 13:22:44.174771 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:44.174710    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0f635cb2fc3ea0ae61e2f0125780ecd748fe7ac2a09958b3623ec49cd8ee723"
Jul 21 13:22:50.994875 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:50.994836    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"]
Jul 21 13:22:50.995426 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:50.995088    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b" podUID="cdbc746e-708d-4871-8a3a-a9ecec02b661" containerName="maas-api" containerID="cri-o://603cb360237814c963bd48d3312d8b2119d25ec4338fe16d2de3d9a796c8d66c" gracePeriod=30
Jul 21 13:22:51.019813 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:51.019783    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:51.208509 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:51.208475    2563 generic.go:356] "Generic (PLEG): container finished" podID="cdbc746e-708d-4871-8a3a-a9ecec02b661" containerID="603cb360237814c963bd48d3312d8b2119d25ec4338fe16d2de3d9a796c8d66c" exitCode=0
Jul 21 13:22:51.208682 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:51.208535    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b" event={"ID":"cdbc746e-708d-4871-8a3a-a9ecec02b661","Type":"ContainerDied","Data":"603cb360237814c963bd48d3312d8b2119d25ec4338fe16d2de3d9a796c8d66c"}
Jul 21 13:22:51.254055 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:51.254030    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:51.382521 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:51.382484    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbc746e-708d-4871-8a3a-a9ecec02b661-openshift-service-ca\") pod \"cdbc746e-708d-4871-8a3a-a9ecec02b661\" (UID: \"cdbc746e-708d-4871-8a3a-a9ecec02b661\") "
Jul 21 13:22:51.382521 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:51.382530    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jqlb\" (UniqueName: \"kubernetes.io/projected/cdbc746e-708d-4871-8a3a-a9ecec02b661-kube-api-access-4jqlb\") pod \"cdbc746e-708d-4871-8a3a-a9ecec02b661\" (UID: \"cdbc746e-708d-4871-8a3a-a9ecec02b661\") "
Jul 21 13:22:51.382783 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:51.382558    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbc746e-708d-4871-8a3a-a9ecec02b661-maas-api-tls\") pod \"cdbc746e-708d-4871-8a3a-a9ecec02b661\" (UID: \"cdbc746e-708d-4871-8a3a-a9ecec02b661\") "
Jul 21 13:22:51.382923 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:51.382896    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdbc746e-708d-4871-8a3a-a9ecec02b661-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cdbc746e-708d-4871-8a3a-a9ecec02b661" (UID: "cdbc746e-708d-4871-8a3a-a9ecec02b661"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:22:51.384876 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:51.384850    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdbc746e-708d-4871-8a3a-a9ecec02b661-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cdbc746e-708d-4871-8a3a-a9ecec02b661" (UID: "cdbc746e-708d-4871-8a3a-a9ecec02b661"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:22:51.384986 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:51.384890    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdbc746e-708d-4871-8a3a-a9ecec02b661-kube-api-access-4jqlb" (OuterVolumeSpecName: "kube-api-access-4jqlb") pod "cdbc746e-708d-4871-8a3a-a9ecec02b661" (UID: "cdbc746e-708d-4871-8a3a-a9ecec02b661"). InnerVolumeSpecName "kube-api-access-4jqlb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:22:51.483247 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:51.483202    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdbc746e-708d-4871-8a3a-a9ecec02b661-openshift-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:22:51.483247 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:51.483241    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jqlb\" (UniqueName: \"kubernetes.io/projected/cdbc746e-708d-4871-8a3a-a9ecec02b661-kube-api-access-4jqlb\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:22:51.483247 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:51.483253    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdbc746e-708d-4871-8a3a-a9ecec02b661-maas-api-tls\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:22:52.214624 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:52.214590    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b" event={"ID":"cdbc746e-708d-4871-8a3a-a9ecec02b661","Type":"ContainerDied","Data":"c1eaa551309effeffb8939cd0ed5b7263333397976c68a589578aa1607f78991"}
Jul 21 13:22:52.214624 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:52.214624    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"
Jul 21 13:22:52.215137 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:52.214652    2563 scope.go:117] "RemoveContainer" containerID="603cb360237814c963bd48d3312d8b2119d25ec4338fe16d2de3d9a796c8d66c"
Jul 21 13:22:52.231532 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:52.231499    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"]
Jul 21 13:22:52.235432 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:52.235403    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ea9fc8fd-99f8f6f65-c479b"]
Jul 21 13:22:53.995634 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:53.995600    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdbc746e-708d-4871-8a3a-a9ecec02b661" path="/var/lib/kubelet/pods/cdbc746e-708d-4871-8a3a-a9ecec02b661/volumes"
Jul 21 13:22:55.662382 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:55.662347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:22:55.667317 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:55.667275    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5"]
Jul 21 13:22:55.669637 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:55.669613    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea9fc8fd-qskm5"]
Jul 21 13:22:55.995214 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:55.995132    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99312641-bc52-420d-b730-9161c95dff0a" path="/var/lib/kubelet/pods/99312641-bc52-420d-b730-9161c95dff0a/volumes"
Jul 21 13:22:56.125434 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:22:56.125402    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:01.477965 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:01.477932    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:02.031503 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:02.031465    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:02.098574 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:02.098542    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:02.330384 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:02.330269    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:02.583471 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:02.583383    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:02.748223 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:02.748160    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:02.852161 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:02.852136    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:02.974717 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:02.974672    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:08.871742 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:08.871707    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:08.986635 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:08.986603    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:09.078684 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:09.078647    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:09.264251 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:09.264162    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:14.787616 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:14.787578    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:14.888237 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:14.888196    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:22.053979 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:22.053944    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:22.370056 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:22.370006    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:22.468867 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:22.468835    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:25.820555 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.820517    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:25.904715 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.904679    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"]
Jul 21 13:23:25.905010 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.904996    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99312641-bc52-420d-b730-9161c95dff0a" containerName="revoke-keys"
Jul 21 13:23:25.905073 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.905011    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="99312641-bc52-420d-b730-9161c95dff0a" containerName="revoke-keys"
Jul 21 13:23:25.905073 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.905031    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdbc746e-708d-4871-8a3a-a9ecec02b661" containerName="maas-api"
Jul 21 13:23:25.905073 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.905036    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdbc746e-708d-4871-8a3a-a9ecec02b661" containerName="maas-api"
Jul 21 13:23:25.905225 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.905094    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="99312641-bc52-420d-b730-9161c95dff0a" containerName="revoke-keys"
Jul 21 13:23:25.905225 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.905104    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdbc746e-708d-4871-8a3a-a9ecec02b661" containerName="maas-api"
Jul 21 13:23:25.908197 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.908174    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:25.911539 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.911512    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 13:23:25.911658 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.911615    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-eee3d688-serving-cert\""
Jul 21 13:23:25.911853 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.911838    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 13:23:25.912684 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.912667    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pkltq\""
Jul 21 13:23:25.924084 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.923892    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:25.926476 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.926449    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"]
Jul 21 13:23:25.993751 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.993717    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1ec3930-6329-4345-841b-ed6505a8dbf8-maas-api-tls\") pod \"maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8\" (UID: \"d1ec3930-6329-4345-841b-ed6505a8dbf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:25.993920 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.993834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c22pk\" (UniqueName: \"kubernetes.io/projected/d1ec3930-6329-4345-841b-ed6505a8dbf8-kube-api-access-c22pk\") pod \"maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8\" (UID: \"d1ec3930-6329-4345-841b-ed6505a8dbf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:25.993920 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:25.993906    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1ec3930-6329-4345-841b-ed6505a8dbf8-openshift-service-ca\") pod \"maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8\" (UID: \"d1ec3930-6329-4345-841b-ed6505a8dbf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:26.025870 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:26.025840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:26.094606 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:26.094506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1ec3930-6329-4345-841b-ed6505a8dbf8-openshift-service-ca\") pod \"maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8\" (UID: \"d1ec3930-6329-4345-841b-ed6505a8dbf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:26.094606 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:26.094547    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1ec3930-6329-4345-841b-ed6505a8dbf8-maas-api-tls\") pod \"maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8\" (UID: \"d1ec3930-6329-4345-841b-ed6505a8dbf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:26.094840 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:26.094624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c22pk\" (UniqueName: \"kubernetes.io/projected/d1ec3930-6329-4345-841b-ed6505a8dbf8-kube-api-access-c22pk\") pod \"maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8\" (UID: \"d1ec3930-6329-4345-841b-ed6505a8dbf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:26.094840 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:23:26.094719    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-serving-cert: secret "maas-api-e2e-mt-eee3d688-serving-cert" not found
Jul 21 13:23:26.094840 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:23:26.094796    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1ec3930-6329-4345-841b-ed6505a8dbf8-maas-api-tls podName:d1ec3930-6329-4345-841b-ed6505a8dbf8 nodeName:}" failed. No retries permitted until 2026-07-21 13:23:26.594779735 +0000 UTC m=+1939.120282872 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d1ec3930-6329-4345-841b-ed6505a8dbf8-maas-api-tls") pod "maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8" (UID: "d1ec3930-6329-4345-841b-ed6505a8dbf8") : secret "maas-api-e2e-mt-eee3d688-serving-cert" not found
Jul 21 13:23:26.095236 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:26.095218    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1ec3930-6329-4345-841b-ed6505a8dbf8-openshift-service-ca\") pod \"maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8\" (UID: \"d1ec3930-6329-4345-841b-ed6505a8dbf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:26.106145 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:26.106125    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c22pk\" (UniqueName: \"kubernetes.io/projected/d1ec3930-6329-4345-841b-ed6505a8dbf8-kube-api-access-c22pk\") pod \"maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8\" (UID: \"d1ec3930-6329-4345-841b-ed6505a8dbf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:26.504499 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:26.504466    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:26.598839 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:26.598794    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1ec3930-6329-4345-841b-ed6505a8dbf8-maas-api-tls\") pod \"maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8\" (UID: \"d1ec3930-6329-4345-841b-ed6505a8dbf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:26.601485 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:26.601462    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1ec3930-6329-4345-841b-ed6505a8dbf8-maas-api-tls\") pod \"maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8\" (UID: \"d1ec3930-6329-4345-841b-ed6505a8dbf8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:26.602253 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:26.602232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:26.823279 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:26.823191    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:26.958441 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:26.958324    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"]
Jul 21 13:23:26.961220 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:23:26.961181    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1ec3930_6329_4345_841b_ed6505a8dbf8.slice/crio-410dbf1e54a5111a3a75b59da8faa997b4a3eb5307c4473c93bdc598fae46e93 WatchSource:0}: Error finding container 410dbf1e54a5111a3a75b59da8faa997b4a3eb5307c4473c93bdc598fae46e93: Status 404 returned error can't find the container with id 410dbf1e54a5111a3a75b59da8faa997b4a3eb5307c4473c93bdc598fae46e93
Jul 21 13:23:27.366625 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:27.366598    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8" event={"ID":"d1ec3930-6329-4345-841b-ed6505a8dbf8","Type":"ContainerStarted","Data":"410dbf1e54a5111a3a75b59da8faa997b4a3eb5307c4473c93bdc598fae46e93"}
Jul 21 13:23:28.371867 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:28.371822    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8" event={"ID":"d1ec3930-6329-4345-841b-ed6505a8dbf8","Type":"ContainerStarted","Data":"aacb1ee309634d2ab9fb8356e58c17577b469c120d29973eedc35c7b44a23039"}
Jul 21 13:23:28.372276 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:28.371896    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:28.391395 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:28.391340    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8" podStartSLOduration=2.994735069 podStartE2EDuration="3.391323298s" totalImagesPullingTime="396.588229ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:23:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:23:28.389354211 +0000 UTC m=+1940.914857366" watchObservedRunningTime="2026-07-21 13:23:28.391323298 +0000 UTC m=+1940.916826477"
Jul 21 13:23:30.408435 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:30.408393    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:34.381691 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:34.381664    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:23:47.313818 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.313786    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"]
Jul 21 13:23:47.317400 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.317316    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.320430 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.320395    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-eee3d688\"/\"openshift-service-ca.crt\""
Jul 21 13:23:47.321307 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.321271    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-eee3d688\"/\"e2e-lifecycle-model-eee3d688-kserve-self-signed-certs\""
Jul 21 13:23:47.321672 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.321655    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-eee3d688\"/\"default-dockercfg-z8vs7\""
Jul 21 13:23:47.321757 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.321693    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-eee3d688\"/\"kube-root-ca.crt\""
Jul 21 13:23:47.328225 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.328202    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"]
Jul 21 13:23:47.442474 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.442446    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:23:47.485860 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.485814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-home\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.486037 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.485888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-tmp-dir\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.486037 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.485987    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-dshm\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.486037 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.486014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghwmj\" (UniqueName: \"kubernetes.io/projected/b9ebe991-c77a-4686-b19b-4a66c99dcb70-kube-api-access-ghwmj\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.486163 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.486043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-model-cache\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.486163 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.486072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9ebe991-c77a-4686-b19b-4a66c99dcb70-tls-certs\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.586826 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.586731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-dshm\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.586826 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.586772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghwmj\" (UniqueName: \"kubernetes.io/projected/b9ebe991-c77a-4686-b19b-4a66c99dcb70-kube-api-access-ghwmj\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.586826 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.586795    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-model-cache\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.587062 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.586919    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9ebe991-c77a-4686-b19b-4a66c99dcb70-tls-certs\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.587062 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.586987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-home\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.587163 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.587071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-tmp-dir\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.587219 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.587191    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-model-cache\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.587409 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.587386    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-home\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.587494 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.587442    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-tmp-dir\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.589226 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.589205    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-dshm\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.589499 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.589483    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9ebe991-c77a-4686-b19b-4a66c99dcb70-tls-certs\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.595998 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.595970    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghwmj\" (UniqueName: \"kubernetes.io/projected/b9ebe991-c77a-4686-b19b-4a66c99dcb70-kube-api-access-ghwmj\") pod \"e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") " pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.627805 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.627765    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:47.760162 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:47.760129    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"]
Jul 21 13:23:47.762623 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:23:47.762596    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9ebe991_c77a_4686_b19b_4a66c99dcb70.slice/crio-86f8d47ce3f8a614bfe12d172e1db376430226dc7f80f598b076b17a2b1bd9c7 WatchSource:0}: Error finding container 86f8d47ce3f8a614bfe12d172e1db376430226dc7f80f598b076b17a2b1bd9c7: Status 404 returned error can't find the container with id 86f8d47ce3f8a614bfe12d172e1db376430226dc7f80f598b076b17a2b1bd9c7
Jul 21 13:23:48.459502 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:48.459464    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk" event={"ID":"b9ebe991-c77a-4686-b19b-4a66c99dcb70","Type":"ContainerStarted","Data":"60322bd6e57cc5dcbb2805f922b40b54961a2d678edab3c52d6d5a2f16d6dc94"}
Jul 21 13:23:48.459502 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:48.459511    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk" event={"ID":"b9ebe991-c77a-4686-b19b-4a66c99dcb70","Type":"ContainerStarted","Data":"86f8d47ce3f8a614bfe12d172e1db376430226dc7f80f598b076b17a2b1bd9c7"}
Jul 21 13:23:48.478459 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:48.478412    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk" podStartSLOduration=1.478396794 podStartE2EDuration="1.478396794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:23:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:23:48.476006503 +0000 UTC m=+1961.001509657" watchObservedRunningTime="2026-07-21 13:23:48.478396794 +0000 UTC m=+1961.003899949"
Jul 21 13:23:57.628761 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:57.628713    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:57.629271 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:57.628785    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:57.640844 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:57.640815    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:23:58.512712 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:23:58.512676    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:24:14.345329 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:14.345279    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc"]
Jul 21 13:24:14.348468 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:14.348452    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc"
Jul 21 13:24:14.350992 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:14.350968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qhp7l\""
Jul 21 13:24:14.355824 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:14.355712    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc"]
Jul 21 13:24:14.413085 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:14.413053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61e97eac-838f-4ad9-b84d-3f29ac41ef79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc\" (UID: \"61e97eac-838f-4ad9-b84d-3f29ac41ef79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc"
Jul 21 13:24:14.514658 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:14.514615    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61e97eac-838f-4ad9-b84d-3f29ac41ef79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc\" (UID: \"61e97eac-838f-4ad9-b84d-3f29ac41ef79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc"
Jul 21 13:24:14.515235 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:14.515213    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61e97eac-838f-4ad9-b84d-3f29ac41ef79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc\" (UID: \"61e97eac-838f-4ad9-b84d-3f29ac41ef79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc"
Jul 21 13:24:14.659454 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:14.659404    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc"
Jul 21 13:24:14.792166 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:14.792138    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc"]
Jul 21 13:24:14.794032 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:24:14.793993    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61e97eac_838f_4ad9_b84d_3f29ac41ef79.slice/crio-93c0a87b0d99b707fa04b413233c1f4808f5ce2082d6463eccc884528a8c74c4 WatchSource:0}: Error finding container 93c0a87b0d99b707fa04b413233c1f4808f5ce2082d6463eccc884528a8c74c4: Status 404 returned error can't find the container with id 93c0a87b0d99b707fa04b413233c1f4808f5ce2082d6463eccc884528a8c74c4
Jul 21 13:24:15.575779 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:15.575741    2563 generic.go:356] "Generic (PLEG): container finished" podID="61e97eac-838f-4ad9-b84d-3f29ac41ef79" containerID="891af4f67122c90962ca18f63463f339f6f1c54f9b319f0a9bf32bfe51d51bc9" exitCode=0
Jul 21 13:24:15.576186 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:15.575804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc" event={"ID":"61e97eac-838f-4ad9-b84d-3f29ac41ef79","Type":"ContainerDied","Data":"891af4f67122c90962ca18f63463f339f6f1c54f9b319f0a9bf32bfe51d51bc9"}
Jul 21 13:24:15.576186 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:15.575844    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc" event={"ID":"61e97eac-838f-4ad9-b84d-3f29ac41ef79","Type":"ContainerStarted","Data":"93c0a87b0d99b707fa04b413233c1f4808f5ce2082d6463eccc884528a8c74c4"}
Jul 21 13:24:16.711501 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:16.711468    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc"
Jul 21 13:24:16.733843 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:16.733807    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61e97eac-838f-4ad9-b84d-3f29ac41ef79-maas-api-service-ca\") pod \"61e97eac-838f-4ad9-b84d-3f29ac41ef79\" (UID: \"61e97eac-838f-4ad9-b84d-3f29ac41ef79\") "
Jul 21 13:24:16.734148 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:16.734125    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61e97eac-838f-4ad9-b84d-3f29ac41ef79-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "61e97eac-838f-4ad9-b84d-3f29ac41ef79" (UID: "61e97eac-838f-4ad9-b84d-3f29ac41ef79"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:24:16.834937 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:16.834900    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61e97eac-838f-4ad9-b84d-3f29ac41ef79-maas-api-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:24:17.586472 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:17.586438    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc"
Jul 21 13:24:17.586648 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:17.586471    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc" event={"ID":"61e97eac-838f-4ad9-b84d-3f29ac41ef79","Type":"ContainerDied","Data":"93c0a87b0d99b707fa04b413233c1f4808f5ce2082d6463eccc884528a8c74c4"}
Jul 21 13:24:17.586648 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:17.586518    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="93c0a87b0d99b707fa04b413233c1f4808f5ce2082d6463eccc884528a8c74c4"
Jul 21 13:24:29.782692 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:29.782657    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"]
Jul 21 13:24:29.785287 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:29.782873    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8" podUID="d1ec3930-6329-4345-841b-ed6505a8dbf8" containerName="maas-api" containerID="cri-o://aacb1ee309634d2ab9fb8356e58c17577b469c120d29973eedc35c7b44a23039" gracePeriod=30
Jul 21 13:24:29.953671 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:29.953592    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:24:30.025833 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.025809    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:24:30.150579 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.150532    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c22pk\" (UniqueName: \"kubernetes.io/projected/d1ec3930-6329-4345-841b-ed6505a8dbf8-kube-api-access-c22pk\") pod \"d1ec3930-6329-4345-841b-ed6505a8dbf8\" (UID: \"d1ec3930-6329-4345-841b-ed6505a8dbf8\") "
Jul 21 13:24:30.150579 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.150585    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1ec3930-6329-4345-841b-ed6505a8dbf8-openshift-service-ca\") pod \"d1ec3930-6329-4345-841b-ed6505a8dbf8\" (UID: \"d1ec3930-6329-4345-841b-ed6505a8dbf8\") "
Jul 21 13:24:30.150850 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.150677    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1ec3930-6329-4345-841b-ed6505a8dbf8-maas-api-tls\") pod \"d1ec3930-6329-4345-841b-ed6505a8dbf8\" (UID: \"d1ec3930-6329-4345-841b-ed6505a8dbf8\") "
Jul 21 13:24:30.151046 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.151014    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1ec3930-6329-4345-841b-ed6505a8dbf8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d1ec3930-6329-4345-841b-ed6505a8dbf8" (UID: "d1ec3930-6329-4345-841b-ed6505a8dbf8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:24:30.152773 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.152748    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1ec3930-6329-4345-841b-ed6505a8dbf8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d1ec3930-6329-4345-841b-ed6505a8dbf8" (UID: "d1ec3930-6329-4345-841b-ed6505a8dbf8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:24:30.152875 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.152848    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1ec3930-6329-4345-841b-ed6505a8dbf8-kube-api-access-c22pk" (OuterVolumeSpecName: "kube-api-access-c22pk") pod "d1ec3930-6329-4345-841b-ed6505a8dbf8" (UID: "d1ec3930-6329-4345-841b-ed6505a8dbf8"). InnerVolumeSpecName "kube-api-access-c22pk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:24:30.251765 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.251733    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1ec3930-6329-4345-841b-ed6505a8dbf8-maas-api-tls\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:24:30.251765 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.251760    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c22pk\" (UniqueName: \"kubernetes.io/projected/d1ec3930-6329-4345-841b-ed6505a8dbf8-kube-api-access-c22pk\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:24:30.251765 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.251770    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1ec3930-6329-4345-841b-ed6505a8dbf8-openshift-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:24:30.645601 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.645563    2563 generic.go:356] "Generic (PLEG): container finished" podID="d1ec3930-6329-4345-841b-ed6505a8dbf8" containerID="aacb1ee309634d2ab9fb8356e58c17577b469c120d29973eedc35c7b44a23039" exitCode=0
Jul 21 13:24:30.645801 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.645616    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8" event={"ID":"d1ec3930-6329-4345-841b-ed6505a8dbf8","Type":"ContainerDied","Data":"aacb1ee309634d2ab9fb8356e58c17577b469c120d29973eedc35c7b44a23039"}
Jul 21 13:24:30.645801 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.645625    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"
Jul 21 13:24:30.645801 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.645685    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8" event={"ID":"d1ec3930-6329-4345-841b-ed6505a8dbf8","Type":"ContainerDied","Data":"410dbf1e54a5111a3a75b59da8faa997b4a3eb5307c4473c93bdc598fae46e93"}
Jul 21 13:24:30.645801 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.645709    2563 scope.go:117] "RemoveContainer" containerID="aacb1ee309634d2ab9fb8356e58c17577b469c120d29973eedc35c7b44a23039"
Jul 21 13:24:30.654778 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.654759    2563 scope.go:117] "RemoveContainer" containerID="aacb1ee309634d2ab9fb8356e58c17577b469c120d29973eedc35c7b44a23039"
Jul 21 13:24:30.655038 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:24:30.655009    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aacb1ee309634d2ab9fb8356e58c17577b469c120d29973eedc35c7b44a23039\": container with ID starting with aacb1ee309634d2ab9fb8356e58c17577b469c120d29973eedc35c7b44a23039 not found: ID does not exist" containerID="aacb1ee309634d2ab9fb8356e58c17577b469c120d29973eedc35c7b44a23039"
Jul 21 13:24:30.655111 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.655036    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacb1ee309634d2ab9fb8356e58c17577b469c120d29973eedc35c7b44a23039"} err="failed to get container status \"aacb1ee309634d2ab9fb8356e58c17577b469c120d29973eedc35c7b44a23039\": rpc error: code = NotFound desc = could not find container \"aacb1ee309634d2ab9fb8356e58c17577b469c120d29973eedc35c7b44a23039\": container with ID starting with aacb1ee309634d2ab9fb8356e58c17577b469c120d29973eedc35c7b44a23039 not found: ID does not exist"
Jul 21 13:24:30.667848 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.667822    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"]
Jul 21 13:24:30.671004 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:30.670981    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-eee3d688-7db5f7485c-gdrk8"]
Jul 21 13:24:31.995521 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:31.995487    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1ec3930-6329-4345-841b-ed6505a8dbf8" path="/var/lib/kubelet/pods/d1ec3930-6329-4345-841b-ed6505a8dbf8/volumes"
Jul 21 13:24:34.428052 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:34.428017    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc"]
Jul 21 13:24:34.429586 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:34.429563    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-eee3d688-wcbsc"]
Jul 21 13:24:34.434880 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:34.434847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:24:35.995046 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:35.995010    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61e97eac-838f-4ad9-b84d-3f29ac41ef79" path="/var/lib/kubelet/pods/61e97eac-838f-4ad9-b84d-3f29ac41ef79/volumes"
Jul 21 13:24:36.981201 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:36.981164    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:24:41.965087 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:41.965051    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"]
Jul 21 13:24:42.074500 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:42.074466    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:24:56.986459 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:56.986412    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk" podUID="b9ebe991-c77a-4686-b19b-4a66c99dcb70" containerName="main" containerID="cri-o://60322bd6e57cc5dcbb2805f922b40b54961a2d678edab3c52d6d5a2f16d6dc94" gracePeriod=45
Jul 21 13:24:57.226370 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.226345    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:24:57.303481 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.303404    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghwmj\" (UniqueName: \"kubernetes.io/projected/b9ebe991-c77a-4686-b19b-4a66c99dcb70-kube-api-access-ghwmj\") pod \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") "
Jul 21 13:24:57.303481 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.303454    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-dshm\") pod \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") "
Jul 21 13:24:57.303481 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.303478    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9ebe991-c77a-4686-b19b-4a66c99dcb70-tls-certs\") pod \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") "
Jul 21 13:24:57.303732 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.303519    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-model-cache\") pod \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") "
Jul 21 13:24:57.303732 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.303563    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-home\") pod \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") "
Jul 21 13:24:57.303732 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.303586    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-tmp-dir\") pod \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\" (UID: \"b9ebe991-c77a-4686-b19b-4a66c99dcb70\") "
Jul 21 13:24:57.303885 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.303816    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-model-cache" (OuterVolumeSpecName: "model-cache") pod "b9ebe991-c77a-4686-b19b-4a66c99dcb70" (UID: "b9ebe991-c77a-4686-b19b-4a66c99dcb70"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:24:57.303885 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.303870    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-home" (OuterVolumeSpecName: "home") pod "b9ebe991-c77a-4686-b19b-4a66c99dcb70" (UID: "b9ebe991-c77a-4686-b19b-4a66c99dcb70"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:24:57.303966 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.303914    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b9ebe991-c77a-4686-b19b-4a66c99dcb70" (UID: "b9ebe991-c77a-4686-b19b-4a66c99dcb70"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:24:57.305682 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.305659    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-dshm" (OuterVolumeSpecName: "dshm") pod "b9ebe991-c77a-4686-b19b-4a66c99dcb70" (UID: "b9ebe991-c77a-4686-b19b-4a66c99dcb70"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:24:57.305780 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.305740    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9ebe991-c77a-4686-b19b-4a66c99dcb70-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b9ebe991-c77a-4686-b19b-4a66c99dcb70" (UID: "b9ebe991-c77a-4686-b19b-4a66c99dcb70"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:24:57.305822 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.305804    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9ebe991-c77a-4686-b19b-4a66c99dcb70-kube-api-access-ghwmj" (OuterVolumeSpecName: "kube-api-access-ghwmj") pod "b9ebe991-c77a-4686-b19b-4a66c99dcb70" (UID: "b9ebe991-c77a-4686-b19b-4a66c99dcb70"). InnerVolumeSpecName "kube-api-access-ghwmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:24:57.404524 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.404489    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghwmj\" (UniqueName: \"kubernetes.io/projected/b9ebe991-c77a-4686-b19b-4a66c99dcb70-kube-api-access-ghwmj\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:24:57.404524 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.404523    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-dshm\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:24:57.404524 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.404532    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9ebe991-c77a-4686-b19b-4a66c99dcb70-tls-certs\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:24:57.404762 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.404542    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-model-cache\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:24:57.404762 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.404549    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-home\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:24:57.404762 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.404557    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9ebe991-c77a-4686-b19b-4a66c99dcb70-tmp-dir\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:24:57.770691 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.770656    2563 generic.go:356] "Generic (PLEG): container finished" podID="b9ebe991-c77a-4686-b19b-4a66c99dcb70" containerID="60322bd6e57cc5dcbb2805f922b40b54961a2d678edab3c52d6d5a2f16d6dc94" exitCode=0
Jul 21 13:24:57.770871 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.770706    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"
Jul 21 13:24:57.770871 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.770737    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk" event={"ID":"b9ebe991-c77a-4686-b19b-4a66c99dcb70","Type":"ContainerDied","Data":"60322bd6e57cc5dcbb2805f922b40b54961a2d678edab3c52d6d5a2f16d6dc94"}
Jul 21 13:24:57.770871 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.770780    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk" event={"ID":"b9ebe991-c77a-4686-b19b-4a66c99dcb70","Type":"ContainerDied","Data":"86f8d47ce3f8a614bfe12d172e1db376430226dc7f80f598b076b17a2b1bd9c7"}
Jul 21 13:24:57.770871 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.770798    2563 scope.go:117] "RemoveContainer" containerID="60322bd6e57cc5dcbb2805f922b40b54961a2d678edab3c52d6d5a2f16d6dc94"
Jul 21 13:24:57.780447 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.780431    2563 scope.go:117] "RemoveContainer" containerID="60322bd6e57cc5dcbb2805f922b40b54961a2d678edab3c52d6d5a2f16d6dc94"
Jul 21 13:24:57.780724 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:24:57.780688    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60322bd6e57cc5dcbb2805f922b40b54961a2d678edab3c52d6d5a2f16d6dc94\": container with ID starting with 60322bd6e57cc5dcbb2805f922b40b54961a2d678edab3c52d6d5a2f16d6dc94 not found: ID does not exist" containerID="60322bd6e57cc5dcbb2805f922b40b54961a2d678edab3c52d6d5a2f16d6dc94"
Jul 21 13:24:57.780781 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.780731    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60322bd6e57cc5dcbb2805f922b40b54961a2d678edab3c52d6d5a2f16d6dc94"} err="failed to get container status \"60322bd6e57cc5dcbb2805f922b40b54961a2d678edab3c52d6d5a2f16d6dc94\": rpc error: code = NotFound desc = could not find container \"60322bd6e57cc5dcbb2805f922b40b54961a2d678edab3c52d6d5a2f16d6dc94\": container with ID starting with 60322bd6e57cc5dcbb2805f922b40b54961a2d678edab3c52d6d5a2f16d6dc94 not found: ID does not exist"
Jul 21 13:24:57.793356 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.793327    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"]
Jul 21 13:24:57.797520 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.797499    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-eee3d688/e2e-lifecycle-model-eee3d688-kserve-699956896c-5bqlk"]
Jul 21 13:24:58.000140 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:24:57.997217    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9ebe991-c77a-4686-b19b-4a66c99dcb70" path="/var/lib/kubelet/pods/b9ebe991-c77a-4686-b19b-4a66c99dcb70/volumes"
Jul 21 13:25:07.651008 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:07.650974    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:09.138475 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:09.138435    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:18.823269 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:18.823232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:19.320616 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:19.320581    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:24.928120 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:24.928093    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:25.027881 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:25.027850    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:25.245504 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:25.245411    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:25.531610 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:25.531516    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:26.640985 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:26.640952    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:26.722524 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:26.722488    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:26.822709 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:26.822678    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:27.661782 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:27.661751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:27.798442 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:27.798406    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:29.319461 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:29.319426    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:35.145973 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:35.145936    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:35.265791 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:35.265751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:35.853541 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:35.853505    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:41.859428 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:41.859394    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:41.954783 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:41.954744    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:42.033073 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:42.033040    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:42.722449 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:42.722421    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:42.826825 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:42.826788    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:52.573874 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:52.573840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:25:52.915861 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:25:52.915822    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:02.236287 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:02.236252    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:08.068902 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:08.068867    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:26:08.075106 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:08.075084    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:26:13.896234 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:13.896198    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:20.719511 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:20.719476    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:20.819137 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:20.819088    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:21.025145 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:21.025065    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:21.362100 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:21.362072    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:21.473649 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:21.473611    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:24.763521 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.763483    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:24.862789 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.862749    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:24.921564 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.921527    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"]
Jul 21 13:26:24.922020 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.922005    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9ebe991-c77a-4686-b19b-4a66c99dcb70" containerName="main"
Jul 21 13:26:24.922072 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.922025    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9ebe991-c77a-4686-b19b-4a66c99dcb70" containerName="main"
Jul 21 13:26:24.922072 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.922042    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1ec3930-6329-4345-841b-ed6505a8dbf8" containerName="maas-api"
Jul 21 13:26:24.922072 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.922050    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1ec3930-6329-4345-841b-ed6505a8dbf8" containerName="maas-api"
Jul 21 13:26:24.922072 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.922066    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61e97eac-838f-4ad9-b84d-3f29ac41ef79" containerName="revoke-keys"
Jul 21 13:26:24.922211 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.922075    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="61e97eac-838f-4ad9-b84d-3f29ac41ef79" containerName="revoke-keys"
Jul 21 13:26:24.922211 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.922149    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1ec3930-6329-4345-841b-ed6505a8dbf8" containerName="maas-api"
Jul 21 13:26:24.922211 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.922160    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="61e97eac-838f-4ad9-b84d-3f29ac41ef79" containerName="revoke-keys"
Jul 21 13:26:24.922211 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.922200    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9ebe991-c77a-4686-b19b-4a66c99dcb70" containerName="main"
Jul 21 13:26:24.925544 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.925524    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:26:24.929975 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.929952    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 13:26:24.930160 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.930147    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pkltq\""
Jul 21 13:26:24.931025 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.931002    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 13:26:24.933818 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.933800    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-d1dafe-serving-cert\""
Jul 21 13:26:24.938593 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.938568    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"]
Jul 21 13:26:24.978543 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:24.978510    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:25.052728 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.052604    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb568f2-4366-4403-aa7c-2911acd43446-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr\" (UID: \"4fb568f2-4366-4403-aa7c-2911acd43446\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:26:25.052728 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.052663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fb568f2-4366-4403-aa7c-2911acd43446-maas-api-tls\") pod \"maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr\" (UID: \"4fb568f2-4366-4403-aa7c-2911acd43446\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:26:25.052979 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.052758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dnbd\" (UniqueName: \"kubernetes.io/projected/4fb568f2-4366-4403-aa7c-2911acd43446-kube-api-access-4dnbd\") pod \"maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr\" (UID: \"4fb568f2-4366-4403-aa7c-2911acd43446\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:26:25.100522 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.100486    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:25.153406 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.153375    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb568f2-4366-4403-aa7c-2911acd43446-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr\" (UID: \"4fb568f2-4366-4403-aa7c-2911acd43446\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:26:25.153572 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.153423    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fb568f2-4366-4403-aa7c-2911acd43446-maas-api-tls\") pod \"maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr\" (UID: \"4fb568f2-4366-4403-aa7c-2911acd43446\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:26:25.153572 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.153446    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dnbd\" (UniqueName: \"kubernetes.io/projected/4fb568f2-4366-4403-aa7c-2911acd43446-kube-api-access-4dnbd\") pod \"maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr\" (UID: \"4fb568f2-4366-4403-aa7c-2911acd43446\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:26:25.153957 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.153935    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb568f2-4366-4403-aa7c-2911acd43446-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr\" (UID: \"4fb568f2-4366-4403-aa7c-2911acd43446\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:26:25.155850 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.155821    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fb568f2-4366-4403-aa7c-2911acd43446-maas-api-tls\") pod \"maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr\" (UID: \"4fb568f2-4366-4403-aa7c-2911acd43446\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:26:25.165028 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.165004    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dnbd\" (UniqueName: \"kubernetes.io/projected/4fb568f2-4366-4403-aa7c-2911acd43446-kube-api-access-4dnbd\") pod \"maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr\" (UID: \"4fb568f2-4366-4403-aa7c-2911acd43446\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:26:25.236033 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.235992    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:26:25.377780 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.377756    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"]
Jul 21 13:26:25.382535 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:26:25.382502    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb568f2_4366_4403_aa7c_2911acd43446.slice/crio-23045eb0774e38cf910f8806754109d55167ee7af72cd54af46a55c419e267bf WatchSource:0}: Error finding container 23045eb0774e38cf910f8806754109d55167ee7af72cd54af46a55c419e267bf: Status 404 returned error can't find the container with id 23045eb0774e38cf910f8806754109d55167ee7af72cd54af46a55c419e267bf
Jul 21 13:26:25.383619 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.383599    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 13:26:25.543650 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.543611    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:25.642211 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:25.642178    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:26.141168 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:26.141137    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr" event={"ID":"4fb568f2-4366-4403-aa7c-2911acd43446","Type":"ContainerStarted","Data":"2d96c3cee387f550f75b69706cfd109ca17f49948ddf2c3ca1545abb14417a18"}
Jul 21 13:26:26.141629 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:26.141186    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr" event={"ID":"4fb568f2-4366-4403-aa7c-2911acd43446","Type":"ContainerStarted","Data":"23045eb0774e38cf910f8806754109d55167ee7af72cd54af46a55c419e267bf"}
Jul 21 13:26:26.141629 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:26.141217    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:26:26.169341 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:26.169267    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr" podStartSLOduration=1.731563339 podStartE2EDuration="2.169252995s" totalImagesPullingTime="437.689656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:26:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:26:26.168359454 +0000 UTC m=+2118.693862608" watchObservedRunningTime="2026-07-21 13:26:26.169252995 +0000 UTC m=+2118.694756150"
Jul 21 13:26:29.235997 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:29.235958    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:29.569023 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:29.568895    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:29.878974 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:29.878921    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:29.977946 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:29.977904    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:32.151529 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:32.151500    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:26:32.809309 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:32.809247    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:32.913927 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:32.913895    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:33.063918 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.063841    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"]
Jul 21 13:26:33.067210 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.067192    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:26:33.070141 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.070119    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ba2d34-serving-cert\""
Jul 21 13:26:33.079218 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.079190    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"]
Jul 21 13:26:33.101078 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.101045    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:33.224595 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.224548    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69fb7f72-02e8-4509-b797-78a5a85d7638-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7\" (UID: \"69fb7f72-02e8-4509-b797-78a5a85d7638\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:26:33.225090 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.224624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69fb7f72-02e8-4509-b797-78a5a85d7638-maas-api-tls\") pod \"maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7\" (UID: \"69fb7f72-02e8-4509-b797-78a5a85d7638\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:26:33.225090 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.224714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxfq2\" (UniqueName: \"kubernetes.io/projected/69fb7f72-02e8-4509-b797-78a5a85d7638-kube-api-access-hxfq2\") pod \"maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7\" (UID: \"69fb7f72-02e8-4509-b797-78a5a85d7638\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:26:33.262679 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.262645    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:33.325790 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.325706    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxfq2\" (UniqueName: \"kubernetes.io/projected/69fb7f72-02e8-4509-b797-78a5a85d7638-kube-api-access-hxfq2\") pod \"maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7\" (UID: \"69fb7f72-02e8-4509-b797-78a5a85d7638\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:26:33.325790 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.325789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69fb7f72-02e8-4509-b797-78a5a85d7638-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7\" (UID: \"69fb7f72-02e8-4509-b797-78a5a85d7638\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:26:33.325974 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.325808    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69fb7f72-02e8-4509-b797-78a5a85d7638-maas-api-tls\") pod \"maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7\" (UID: \"69fb7f72-02e8-4509-b797-78a5a85d7638\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:26:33.326515 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.326492    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69fb7f72-02e8-4509-b797-78a5a85d7638-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7\" (UID: \"69fb7f72-02e8-4509-b797-78a5a85d7638\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:26:33.328319 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.328262    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69fb7f72-02e8-4509-b797-78a5a85d7638-maas-api-tls\") pod \"maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7\" (UID: \"69fb7f72-02e8-4509-b797-78a5a85d7638\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:26:33.338879 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.338859    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxfq2\" (UniqueName: \"kubernetes.io/projected/69fb7f72-02e8-4509-b797-78a5a85d7638-kube-api-access-hxfq2\") pod \"maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7\" (UID: \"69fb7f72-02e8-4509-b797-78a5a85d7638\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:26:33.379659 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.379625    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:26:33.514988 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.514945    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"]
Jul 21 13:26:33.517274 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:26:33.517246    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69fb7f72_02e8_4509_b797_78a5a85d7638.slice/crio-cbb1cd870285b8f030638a12d89376ffe82f5eb14e1b7ebb5fbf0deb2c868482 WatchSource:0}: Error finding container cbb1cd870285b8f030638a12d89376ffe82f5eb14e1b7ebb5fbf0deb2c868482: Status 404 returned error can't find the container with id cbb1cd870285b8f030638a12d89376ffe82f5eb14e1b7ebb5fbf0deb2c868482
Jul 21 13:26:33.592442 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.592355    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:33.693860 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:33.693815    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:34.180883 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:34.180772    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7" event={"ID":"69fb7f72-02e8-4509-b797-78a5a85d7638","Type":"ContainerStarted","Data":"3e8b274b48131d166aad03a9ca7385450b2a2e3ca47a86b7ba10812e7a372baf"}
Jul 21 13:26:34.180883 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:34.180830    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7" event={"ID":"69fb7f72-02e8-4509-b797-78a5a85d7638","Type":"ContainerStarted","Data":"cbb1cd870285b8f030638a12d89376ffe82f5eb14e1b7ebb5fbf0deb2c868482"}
Jul 21 13:26:34.181127 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:34.180935    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:26:34.199521 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:34.199457    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7" podStartSLOduration=0.860932269 podStartE2EDuration="1.199435788s" totalImagesPullingTime="338.503519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:26:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:26:34.198215064 +0000 UTC m=+2126.723718220" watchObservedRunningTime="2026-07-21 13:26:34.199435788 +0000 UTC m=+2126.724938945"
Jul 21 13:26:38.162685 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:38.162653    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:40.190254 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:40.190213    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:26:46.528717 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:46.528687    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:46.630462 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:46.630426    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:54.409544 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:54.409495    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:54.509687 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:54.509653    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:26:58.819154 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:26:58.819071    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:27:08.165056 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:27:08.165020    2563 scope.go:117] "RemoveContainer" containerID="d572d7392864b510c626406f5029f2bfea9f4a66e5d52ebc9df77738c44d62fc"
Jul 21 13:27:18.747109 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:27:18.747071    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:27:24.439739 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:27:24.439705    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:27:31.052468 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:27:31.052438    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:12.212662 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:12.212633    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:13.100375 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:13.100331    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:13.973451 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:13.973418    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:14.074395 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:14.074352    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:23.787034 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:23.786998    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"]
Jul 21 13:28:23.787650 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:23.787237    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr" podUID="4fb568f2-4366-4403-aa7c-2911acd43446" containerName="maas-api" containerID="cri-o://2d96c3cee387f550f75b69706cfd109ca17f49948ddf2c3ca1545abb14417a18" gracePeriod=30
Jul 21 13:28:23.822712 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:23.822674    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:24.034465 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.034434    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:28:24.130193 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.130156    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fb568f2-4366-4403-aa7c-2911acd43446-maas-api-tls\") pod \"4fb568f2-4366-4403-aa7c-2911acd43446\" (UID: \"4fb568f2-4366-4403-aa7c-2911acd43446\") "
Jul 21 13:28:24.130417 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.130239    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb568f2-4366-4403-aa7c-2911acd43446-openshift-service-ca\") pod \"4fb568f2-4366-4403-aa7c-2911acd43446\" (UID: \"4fb568f2-4366-4403-aa7c-2911acd43446\") "
Jul 21 13:28:24.130417 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.130321    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dnbd\" (UniqueName: \"kubernetes.io/projected/4fb568f2-4366-4403-aa7c-2911acd43446-kube-api-access-4dnbd\") pod \"4fb568f2-4366-4403-aa7c-2911acd43446\" (UID: \"4fb568f2-4366-4403-aa7c-2911acd43446\") "
Jul 21 13:28:24.130639 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.130614    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fb568f2-4366-4403-aa7c-2911acd43446-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4fb568f2-4366-4403-aa7c-2911acd43446" (UID: "4fb568f2-4366-4403-aa7c-2911acd43446"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:28:24.137315 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.137218    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fb568f2-4366-4403-aa7c-2911acd43446-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4fb568f2-4366-4403-aa7c-2911acd43446" (UID: "4fb568f2-4366-4403-aa7c-2911acd43446"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:28:24.137804 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.137782    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fb568f2-4366-4403-aa7c-2911acd43446-kube-api-access-4dnbd" (OuterVolumeSpecName: "kube-api-access-4dnbd") pod "4fb568f2-4366-4403-aa7c-2911acd43446" (UID: "4fb568f2-4366-4403-aa7c-2911acd43446"). InnerVolumeSpecName "kube-api-access-4dnbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:28:24.231565 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.231526    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dnbd\" (UniqueName: \"kubernetes.io/projected/4fb568f2-4366-4403-aa7c-2911acd43446-kube-api-access-4dnbd\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:28:24.231565 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.231562    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fb568f2-4366-4403-aa7c-2911acd43446-maas-api-tls\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:28:24.231565 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.231572    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb568f2-4366-4403-aa7c-2911acd43446-openshift-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:28:24.687650 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.687608    2563 generic.go:356] "Generic (PLEG): container finished" podID="4fb568f2-4366-4403-aa7c-2911acd43446" containerID="2d96c3cee387f550f75b69706cfd109ca17f49948ddf2c3ca1545abb14417a18" exitCode=0
Jul 21 13:28:24.687821 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.687677    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"
Jul 21 13:28:24.687821 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.687690    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr" event={"ID":"4fb568f2-4366-4403-aa7c-2911acd43446","Type":"ContainerDied","Data":"2d96c3cee387f550f75b69706cfd109ca17f49948ddf2c3ca1545abb14417a18"}
Jul 21 13:28:24.687821 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.687782    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr" event={"ID":"4fb568f2-4366-4403-aa7c-2911acd43446","Type":"ContainerDied","Data":"23045eb0774e38cf910f8806754109d55167ee7af72cd54af46a55c419e267bf"}
Jul 21 13:28:24.687821 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.687806    2563 scope.go:117] "RemoveContainer" containerID="2d96c3cee387f550f75b69706cfd109ca17f49948ddf2c3ca1545abb14417a18"
Jul 21 13:28:24.696347 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.696326    2563 scope.go:117] "RemoveContainer" containerID="2d96c3cee387f550f75b69706cfd109ca17f49948ddf2c3ca1545abb14417a18"
Jul 21 13:28:24.696618 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:28:24.696598    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d96c3cee387f550f75b69706cfd109ca17f49948ddf2c3ca1545abb14417a18\": container with ID starting with 2d96c3cee387f550f75b69706cfd109ca17f49948ddf2c3ca1545abb14417a18 not found: ID does not exist" containerID="2d96c3cee387f550f75b69706cfd109ca17f49948ddf2c3ca1545abb14417a18"
Jul 21 13:28:24.696678 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.696626    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d96c3cee387f550f75b69706cfd109ca17f49948ddf2c3ca1545abb14417a18"} err="failed to get container status \"2d96c3cee387f550f75b69706cfd109ca17f49948ddf2c3ca1545abb14417a18\": rpc error: code = NotFound desc = could not find container \"2d96c3cee387f550f75b69706cfd109ca17f49948ddf2c3ca1545abb14417a18\": container with ID starting with 2d96c3cee387f550f75b69706cfd109ca17f49948ddf2c3ca1545abb14417a18 not found: ID does not exist"
Jul 21 13:28:24.711031 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.711002    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"]
Jul 21 13:28:24.716949 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:24.716919    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d1dafe-866cd84f8b-p55gr"]
Jul 21 13:28:25.995522 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:25.995492    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fb568f2-4366-4403-aa7c-2911acd43446" path="/var/lib/kubelet/pods/4fb568f2-4366-4403-aa7c-2911acd43446/volumes"
Jul 21 13:28:28.319779 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:28.319675    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:28.735896 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:28.735863    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:34.937835 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:34.937788    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:35.042165 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.042107    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:35.208581 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.208483    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:35.427042 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.427006    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:35.536696 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.536610    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd"]
Jul 21 13:28:35.536947 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.536934    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fb568f2-4366-4403-aa7c-2911acd43446" containerName="maas-api"
Jul 21 13:28:35.536992 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.536949    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fb568f2-4366-4403-aa7c-2911acd43446" containerName="maas-api"
Jul 21 13:28:35.537025 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.537005    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fb568f2-4366-4403-aa7c-2911acd43446" containerName="maas-api"
Jul 21 13:28:35.541227 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.541207    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd"
Jul 21 13:28:35.544341 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.544316    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qhp7l\""
Jul 21 13:28:35.550903 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.550873    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd"]
Jul 21 13:28:35.631546 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.631509    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8742cb8-a7df-4167-9625-4a5292f1e5c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd\" (UID: \"f8742cb8-a7df-4167-9625-4a5292f1e5c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd"
Jul 21 13:28:35.733001 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.732957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8742cb8-a7df-4167-9625-4a5292f1e5c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd\" (UID: \"f8742cb8-a7df-4167-9625-4a5292f1e5c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd"
Jul 21 13:28:35.733625 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.733602    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8742cb8-a7df-4167-9625-4a5292f1e5c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd\" (UID: \"f8742cb8-a7df-4167-9625-4a5292f1e5c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd"
Jul 21 13:28:35.852337 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.852272    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd"
Jul 21 13:28:35.996124 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:35.996099    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd"]
Jul 21 13:28:35.996800 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:28:35.996774    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8742cb8_a7df_4167_9625_4a5292f1e5c3.slice/crio-bf01072cf859a5e008f6a18eb7ec46ebadfece9aecf80995c3ac90a249bcc981 WatchSource:0}: Error finding container bf01072cf859a5e008f6a18eb7ec46ebadfece9aecf80995c3ac90a249bcc981: Status 404 returned error can't find the container with id bf01072cf859a5e008f6a18eb7ec46ebadfece9aecf80995c3ac90a249bcc981
Jul 21 13:28:36.115320 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:36.115201    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:36.216595 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:36.216569    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:36.748553 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:36.748515    2563 generic.go:356] "Generic (PLEG): container finished" podID="f8742cb8-a7df-4167-9625-4a5292f1e5c3" containerID="9d773a623851769574265c1d9742595ac6ee2cf39fb03de265bca0a08681490c" exitCode=0
Jul 21 13:28:36.748752 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:36.748555    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd" event={"ID":"f8742cb8-a7df-4167-9625-4a5292f1e5c3","Type":"ContainerDied","Data":"9d773a623851769574265c1d9742595ac6ee2cf39fb03de265bca0a08681490c"}
Jul 21 13:28:36.748752 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:36.748614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd" event={"ID":"f8742cb8-a7df-4167-9625-4a5292f1e5c3","Type":"ContainerStarted","Data":"bf01072cf859a5e008f6a18eb7ec46ebadfece9aecf80995c3ac90a249bcc981"}
Jul 21 13:28:37.886434 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:37.886413    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd"
Jul 21 13:28:37.955365 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:37.955322    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8742cb8-a7df-4167-9625-4a5292f1e5c3-maas-api-service-ca\") pod \"f8742cb8-a7df-4167-9625-4a5292f1e5c3\" (UID: \"f8742cb8-a7df-4167-9625-4a5292f1e5c3\") "
Jul 21 13:28:37.955715 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:37.955693    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8742cb8-a7df-4167-9625-4a5292f1e5c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f8742cb8-a7df-4167-9625-4a5292f1e5c3" (UID: "f8742cb8-a7df-4167-9625-4a5292f1e5c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:28:38.057179 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:38.057074    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8742cb8-a7df-4167-9625-4a5292f1e5c3-maas-api-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:28:38.761930 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:38.761901    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd"
Jul 21 13:28:38.762153 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:38.761900    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd" event={"ID":"f8742cb8-a7df-4167-9625-4a5292f1e5c3","Type":"ContainerDied","Data":"bf01072cf859a5e008f6a18eb7ec46ebadfece9aecf80995c3ac90a249bcc981"}
Jul 21 13:28:38.762153 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:38.762028    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf01072cf859a5e008f6a18eb7ec46ebadfece9aecf80995c3ac90a249bcc981"
Jul 21 13:28:46.017754 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.017720    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"]
Jul 21 13:28:46.018352 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.017917    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7" podUID="69fb7f72-02e8-4509-b797-78a5a85d7638" containerName="maas-api" containerID="cri-o://3e8b274b48131d166aad03a9ca7385450b2a2e3ca47a86b7ba10812e7a372baf" gracePeriod=30
Jul 21 13:28:46.034790 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.034765    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:46.259225 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.259203    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:28:46.330850 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.330762    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69fb7f72-02e8-4509-b797-78a5a85d7638-openshift-service-ca\") pod \"69fb7f72-02e8-4509-b797-78a5a85d7638\" (UID: \"69fb7f72-02e8-4509-b797-78a5a85d7638\") "
Jul 21 13:28:46.330850 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.330838    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxfq2\" (UniqueName: \"kubernetes.io/projected/69fb7f72-02e8-4509-b797-78a5a85d7638-kube-api-access-hxfq2\") pod \"69fb7f72-02e8-4509-b797-78a5a85d7638\" (UID: \"69fb7f72-02e8-4509-b797-78a5a85d7638\") "
Jul 21 13:28:46.331042 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.330862    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69fb7f72-02e8-4509-b797-78a5a85d7638-maas-api-tls\") pod \"69fb7f72-02e8-4509-b797-78a5a85d7638\" (UID: \"69fb7f72-02e8-4509-b797-78a5a85d7638\") "
Jul 21 13:28:46.331222 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.331198    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69fb7f72-02e8-4509-b797-78a5a85d7638-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "69fb7f72-02e8-4509-b797-78a5a85d7638" (UID: "69fb7f72-02e8-4509-b797-78a5a85d7638"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:28:46.333026 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.333004    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69fb7f72-02e8-4509-b797-78a5a85d7638-kube-api-access-hxfq2" (OuterVolumeSpecName: "kube-api-access-hxfq2") pod "69fb7f72-02e8-4509-b797-78a5a85d7638" (UID: "69fb7f72-02e8-4509-b797-78a5a85d7638"). InnerVolumeSpecName "kube-api-access-hxfq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:28:46.333146 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.333121    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69fb7f72-02e8-4509-b797-78a5a85d7638-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "69fb7f72-02e8-4509-b797-78a5a85d7638" (UID: "69fb7f72-02e8-4509-b797-78a5a85d7638"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:28:46.431666 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.431630    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxfq2\" (UniqueName: \"kubernetes.io/projected/69fb7f72-02e8-4509-b797-78a5a85d7638-kube-api-access-hxfq2\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:28:46.431666 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.431663    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69fb7f72-02e8-4509-b797-78a5a85d7638-maas-api-tls\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:28:46.431666 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.431673    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69fb7f72-02e8-4509-b797-78a5a85d7638-openshift-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:28:46.803360 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.803324    2563 generic.go:356] "Generic (PLEG): container finished" podID="69fb7f72-02e8-4509-b797-78a5a85d7638" containerID="3e8b274b48131d166aad03a9ca7385450b2a2e3ca47a86b7ba10812e7a372baf" exitCode=0
Jul 21 13:28:46.803541 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.803409    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"
Jul 21 13:28:46.803541 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.803420    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7" event={"ID":"69fb7f72-02e8-4509-b797-78a5a85d7638","Type":"ContainerDied","Data":"3e8b274b48131d166aad03a9ca7385450b2a2e3ca47a86b7ba10812e7a372baf"}
Jul 21 13:28:46.803541 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.803473    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7" event={"ID":"69fb7f72-02e8-4509-b797-78a5a85d7638","Type":"ContainerDied","Data":"cbb1cd870285b8f030638a12d89376ffe82f5eb14e1b7ebb5fbf0deb2c868482"}
Jul 21 13:28:46.803541 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.803497    2563 scope.go:117] "RemoveContainer" containerID="3e8b274b48131d166aad03a9ca7385450b2a2e3ca47a86b7ba10812e7a372baf"
Jul 21 13:28:46.812357 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.812341    2563 scope.go:117] "RemoveContainer" containerID="3e8b274b48131d166aad03a9ca7385450b2a2e3ca47a86b7ba10812e7a372baf"
Jul 21 13:28:46.812630 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:28:46.812609    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e8b274b48131d166aad03a9ca7385450b2a2e3ca47a86b7ba10812e7a372baf\": container with ID starting with 3e8b274b48131d166aad03a9ca7385450b2a2e3ca47a86b7ba10812e7a372baf not found: ID does not exist" containerID="3e8b274b48131d166aad03a9ca7385450b2a2e3ca47a86b7ba10812e7a372baf"
Jul 21 13:28:46.812693 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.812643    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e8b274b48131d166aad03a9ca7385450b2a2e3ca47a86b7ba10812e7a372baf"} err="failed to get container status \"3e8b274b48131d166aad03a9ca7385450b2a2e3ca47a86b7ba10812e7a372baf\": rpc error: code = NotFound desc = could not find container \"3e8b274b48131d166aad03a9ca7385450b2a2e3ca47a86b7ba10812e7a372baf\": container with ID starting with 3e8b274b48131d166aad03a9ca7385450b2a2e3ca47a86b7ba10812e7a372baf not found: ID does not exist"
Jul 21 13:28:46.829720 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.829683    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"]
Jul 21 13:28:46.832455 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:46.832433    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba2d34-7d9f5c4496-rg9s7"]
Jul 21 13:28:47.994845 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:47.994814    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69fb7f72-02e8-4509-b797-78a5a85d7638" path="/var/lib/kubelet/pods/69fb7f72-02e8-4509-b797-78a5a85d7638/volumes"
Jul 21 13:28:50.625909 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:50.625872    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd"]
Jul 21 13:28:50.631663 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:50.631635    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2d34-mq5bd"]
Jul 21 13:28:50.643390 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:50.643362    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:51.008466 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:51.008383    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:51.995746 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:51.995710    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8742cb8-a7df-4167-9625-4a5292f1e5c3" path="/var/lib/kubelet/pods/f8742cb8-a7df-4167-9625-4a5292f1e5c3/volumes"
Jul 21 13:28:57.914108 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:57.914055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:57.985421 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:57.985390    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:58.145057 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:58.145024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:58.484918 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:58.484886    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:28:58.909432 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:28:58.909389    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:01.772051 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.772014    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:01.952503 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.952472    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:01.961538 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.961502    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"]
Jul 21 13:29:01.961802 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.961789    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8742cb8-a7df-4167-9625-4a5292f1e5c3" containerName="revoke-keys"
Jul 21 13:29:01.961802 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.961802    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8742cb8-a7df-4167-9625-4a5292f1e5c3" containerName="revoke-keys"
Jul 21 13:29:01.961943 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.961817    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69fb7f72-02e8-4509-b797-78a5a85d7638" containerName="maas-api"
Jul 21 13:29:01.961943 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.961824    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="69fb7f72-02e8-4509-b797-78a5a85d7638" containerName="maas-api"
Jul 21 13:29:01.961943 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.961877    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="69fb7f72-02e8-4509-b797-78a5a85d7638" containerName="maas-api"
Jul 21 13:29:01.961943 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.961884    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8742cb8-a7df-4167-9625-4a5292f1e5c3" containerName="revoke-keys"
Jul 21 13:29:01.964533 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.964516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:01.968388 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.968197    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b6860c-serving-cert\""
Jul 21 13:29:01.968388 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.968218    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 13:29:01.968388 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.968254    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pkltq\""
Jul 21 13:29:01.969514 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.969483    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 13:29:01.981370 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:01.981346    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"]
Jul 21 13:29:02.048830 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.048712    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:02.075070 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.075026    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269968da-d633-4046-8d8d-877fd30a93db-maas-api-tls\") pod \"maas-api-e2e-api-a-b6860c-7cbfd86797-698jq\" (UID: \"269968da-d633-4046-8d8d-877fd30a93db\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:02.075255 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.075096    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwbk5\" (UniqueName: \"kubernetes.io/projected/269968da-d633-4046-8d8d-877fd30a93db-kube-api-access-xwbk5\") pod \"maas-api-e2e-api-a-b6860c-7cbfd86797-698jq\" (UID: \"269968da-d633-4046-8d8d-877fd30a93db\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:02.075255 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.075123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269968da-d633-4046-8d8d-877fd30a93db-openshift-service-ca\") pod \"maas-api-e2e-api-a-b6860c-7cbfd86797-698jq\" (UID: \"269968da-d633-4046-8d8d-877fd30a93db\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:02.176451 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.176414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269968da-d633-4046-8d8d-877fd30a93db-maas-api-tls\") pod \"maas-api-e2e-api-a-b6860c-7cbfd86797-698jq\" (UID: \"269968da-d633-4046-8d8d-877fd30a93db\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:02.176645 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.176467    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwbk5\" (UniqueName: \"kubernetes.io/projected/269968da-d633-4046-8d8d-877fd30a93db-kube-api-access-xwbk5\") pod \"maas-api-e2e-api-a-b6860c-7cbfd86797-698jq\" (UID: \"269968da-d633-4046-8d8d-877fd30a93db\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:02.176645 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.176498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269968da-d633-4046-8d8d-877fd30a93db-openshift-service-ca\") pod \"maas-api-e2e-api-a-b6860c-7cbfd86797-698jq\" (UID: \"269968da-d633-4046-8d8d-877fd30a93db\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:02.177193 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.177172    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269968da-d633-4046-8d8d-877fd30a93db-openshift-service-ca\") pod \"maas-api-e2e-api-a-b6860c-7cbfd86797-698jq\" (UID: \"269968da-d633-4046-8d8d-877fd30a93db\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:02.179169 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.179145    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269968da-d633-4046-8d8d-877fd30a93db-maas-api-tls\") pod \"maas-api-e2e-api-a-b6860c-7cbfd86797-698jq\" (UID: \"269968da-d633-4046-8d8d-877fd30a93db\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:02.185891 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.185868    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwbk5\" (UniqueName: \"kubernetes.io/projected/269968da-d633-4046-8d8d-877fd30a93db-kube-api-access-xwbk5\") pod \"maas-api-e2e-api-a-b6860c-7cbfd86797-698jq\" (UID: \"269968da-d633-4046-8d8d-877fd30a93db\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:02.193981 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.193956    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:02.275470 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.275440    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:02.409147 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.409121    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"]
Jul 21 13:29:02.411261 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:29:02.411228    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod269968da_d633_4046_8d8d_877fd30a93db.slice/crio-eb796623600789f2c0414ab1aab858b4292589d830dae176adfcf08146953257 WatchSource:0}: Error finding container eb796623600789f2c0414ab1aab858b4292589d830dae176adfcf08146953257: Status 404 returned error can't find the container with id eb796623600789f2c0414ab1aab858b4292589d830dae176adfcf08146953257
Jul 21 13:29:02.626963 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.626920    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:02.722710 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.722678    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:02.878112 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:02.878026    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq" event={"ID":"269968da-d633-4046-8d8d-877fd30a93db","Type":"ContainerStarted","Data":"eb796623600789f2c0414ab1aab858b4292589d830dae176adfcf08146953257"}
Jul 21 13:29:03.883338 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:03.883307    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq" event={"ID":"269968da-d633-4046-8d8d-877fd30a93db","Type":"ContainerStarted","Data":"5fd066f9caf95860a0fb1321d46b549e4bf09d3a5a88ab508b7de011312723bf"}
Jul 21 13:29:03.883756 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:03.883459    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:03.904466 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:03.904422    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq" podStartSLOduration=2.417251406 podStartE2EDuration="2.904381102s" totalImagesPullingTime="487.129696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:29:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:29:03.903574151 +0000 UTC m=+2276.429077308" watchObservedRunningTime="2026-07-21 13:29:03.904381102 +0000 UTC m=+2276.429884259"
Jul 21 13:29:06.166114 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:06.166080    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:06.651943 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:06.651914    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:07.076095 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:07.076010    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:07.173162 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:07.173079    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:08.189751 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:08.189719    2563 scope.go:117] "RemoveContainer" containerID="0f21548de73b805f348bc3c50ac7d63400e64453c024830d7bdadd9d620784ea"
Jul 21 13:29:09.213387 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:09.213321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:09.315020 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:09.314968    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:09.748219 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:09.748179    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:09.834745 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:09.834713    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:09.893052 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:09.893024    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:09.938426 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:09.938395    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"]
Jul 21 13:29:09.943332 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:09.943280    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:29:09.947269 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:09.947080    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f114ec-serving-cert\""
Jul 21 13:29:09.961159 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:09.961132    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"]
Jul 21 13:29:10.026910 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.026825    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:10.043185 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.043149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abdac7d4-0bbf-483c-a964-3bd1abaa1920-openshift-service-ca\") pod \"maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj\" (UID: \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:29:10.043185 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.043184    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjlcl\" (UniqueName: \"kubernetes.io/projected/abdac7d4-0bbf-483c-a964-3bd1abaa1920-kube-api-access-rjlcl\") pod \"maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj\" (UID: \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:29:10.043419 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.043333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abdac7d4-0bbf-483c-a964-3bd1abaa1920-maas-api-tls\") pod \"maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj\" (UID: \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:29:10.143840 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.143794    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abdac7d4-0bbf-483c-a964-3bd1abaa1920-maas-api-tls\") pod \"maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj\" (UID: \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:29:10.143840 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.143846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abdac7d4-0bbf-483c-a964-3bd1abaa1920-openshift-service-ca\") pod \"maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj\" (UID: \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:29:10.144104 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.143866    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjlcl\" (UniqueName: \"kubernetes.io/projected/abdac7d4-0bbf-483c-a964-3bd1abaa1920-kube-api-access-rjlcl\") pod \"maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj\" (UID: \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:29:10.144645 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.144621    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abdac7d4-0bbf-483c-a964-3bd1abaa1920-openshift-service-ca\") pod \"maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj\" (UID: \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:29:10.146390 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.146362    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abdac7d4-0bbf-483c-a964-3bd1abaa1920-maas-api-tls\") pod \"maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj\" (UID: \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:29:10.161262 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.161224    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjlcl\" (UniqueName: \"kubernetes.io/projected/abdac7d4-0bbf-483c-a964-3bd1abaa1920-kube-api-access-rjlcl\") pod \"maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj\" (UID: \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:29:10.168561 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.168535    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:10.254982 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.254945    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:29:10.391775 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.391750    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"]
Jul 21 13:29:10.393804 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:29:10.393778    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabdac7d4_0bbf_483c_a964_3bd1abaa1920.slice/crio-67fa2a072d91f749b0d2b672329c1f22f3f3c492ebcab5cd6e30c4af6c09c556 WatchSource:0}: Error finding container 67fa2a072d91f749b0d2b672329c1f22f3f3c492ebcab5cd6e30c4af6c09c556: Status 404 returned error can't find the container with id 67fa2a072d91f749b0d2b672329c1f22f3f3c492ebcab5cd6e30c4af6c09c556
Jul 21 13:29:10.576413 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.576314    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:10.676352 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.676316    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:10.915404 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.915367    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj" event={"ID":"abdac7d4-0bbf-483c-a964-3bd1abaa1920","Type":"ContainerStarted","Data":"cdd4573a68020ca11b282f5d22dfb3ce1eb729985cad98b103a83e695e73e632"}
Jul 21 13:29:10.915585 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.915430    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj" event={"ID":"abdac7d4-0bbf-483c-a964-3bd1abaa1920","Type":"ContainerStarted","Data":"67fa2a072d91f749b0d2b672329c1f22f3f3c492ebcab5cd6e30c4af6c09c556"}
Jul 21 13:29:10.915585 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.915457    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:29:10.935710 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:10.935661    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj" podStartSLOduration=1.494638603 podStartE2EDuration="1.935643137s" totalImagesPullingTime="441.004534ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:29:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:29:10.934646022 +0000 UTC m=+2283.460149179" watchObservedRunningTime="2026-07-21 13:29:10.935643137 +0000 UTC m=+2283.461146316"
Jul 21 13:29:14.190797 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:14.190759    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:16.925261 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:16.925229    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:29:22.088423 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:22.088388    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9"]
Jul 21 13:29:22.091784 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:22.091767    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9"
Jul 21 13:29:22.094429 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:22.094404    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qhp7l\""
Jul 21 13:29:22.108941 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:22.108891    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9"]
Jul 21 13:29:22.155938 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:22.155898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6072b55-c926-4afb-98b3-7991b47f7c1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9\" (UID: \"d6072b55-c926-4afb-98b3-7991b47f7c1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9"
Jul 21 13:29:22.257235 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:22.257195    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6072b55-c926-4afb-98b3-7991b47f7c1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9\" (UID: \"d6072b55-c926-4afb-98b3-7991b47f7c1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9"
Jul 21 13:29:22.257848 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:22.257825    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6072b55-c926-4afb-98b3-7991b47f7c1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9\" (UID: \"d6072b55-c926-4afb-98b3-7991b47f7c1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9"
Jul 21 13:29:22.411154 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:22.411121    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9"
Jul 21 13:29:22.542272 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:22.542242    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9"]
Jul 21 13:29:22.543765 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:29:22.543742    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6072b55_c926_4afb_98b3_7991b47f7c1a.slice/crio-44b73faa83b0be3d46a6eb4b9c229846280fe8a2b55060405a3d3596c273e9ce WatchSource:0}: Error finding container 44b73faa83b0be3d46a6eb4b9c229846280fe8a2b55060405a3d3596c273e9ce: Status 404 returned error can't find the container with id 44b73faa83b0be3d46a6eb4b9c229846280fe8a2b55060405a3d3596c273e9ce
Jul 21 13:29:22.968585 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:22.968542    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9" event={"ID":"d6072b55-c926-4afb-98b3-7991b47f7c1a","Type":"ContainerStarted","Data":"234b76a3b600028cf488a94b428009553a2aaf2af93713b7362fece9854a048b"}
Jul 21 13:29:22.968800 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:22.968603    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9" event={"ID":"d6072b55-c926-4afb-98b3-7991b47f7c1a","Type":"ContainerStarted","Data":"44b73faa83b0be3d46a6eb4b9c229846280fe8a2b55060405a3d3596c273e9ce"}
Jul 21 13:29:22.986154 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:22.986111    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9" podStartSLOduration=0.986096397 podStartE2EDuration="986.096397ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:29:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:29:22.985646494 +0000 UTC m=+2295.511149646" watchObservedRunningTime="2026-07-21 13:29:22.986096397 +0000 UTC m=+2295.511599552"
Jul 21 13:29:23.053673 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:23.053601    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:23.185712 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:23.185680    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:23.974147 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:23.974112    2563 generic.go:356] "Generic (PLEG): container finished" podID="d6072b55-c926-4afb-98b3-7991b47f7c1a" containerID="234b76a3b600028cf488a94b428009553a2aaf2af93713b7362fece9854a048b" exitCode=0
Jul 21 13:29:23.974362 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:23.974169    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9" event={"ID":"d6072b55-c926-4afb-98b3-7991b47f7c1a","Type":"ContainerDied","Data":"234b76a3b600028cf488a94b428009553a2aaf2af93713b7362fece9854a048b"}
Jul 21 13:29:25.107619 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:25.107596    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9"
Jul 21 13:29:25.183536 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:25.183507    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6072b55-c926-4afb-98b3-7991b47f7c1a-maas-api-service-ca\") pod \"d6072b55-c926-4afb-98b3-7991b47f7c1a\" (UID: \"d6072b55-c926-4afb-98b3-7991b47f7c1a\") "
Jul 21 13:29:25.183843 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:25.183822    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6072b55-c926-4afb-98b3-7991b47f7c1a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d6072b55-c926-4afb-98b3-7991b47f7c1a" (UID: "d6072b55-c926-4afb-98b3-7991b47f7c1a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:29:25.284892 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:25.284809    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6072b55-c926-4afb-98b3-7991b47f7c1a-maas-api-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:29:25.984401 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:25.984375    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9"
Jul 21 13:29:25.984401 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:25.984398    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9" event={"ID":"d6072b55-c926-4afb-98b3-7991b47f7c1a","Type":"ContainerDied","Data":"44b73faa83b0be3d46a6eb4b9c229846280fe8a2b55060405a3d3596c273e9ce"}
Jul 21 13:29:25.984598 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:25.984445    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44b73faa83b0be3d46a6eb4b9c229846280fe8a2b55060405a3d3596c273e9ce"
Jul 21 13:29:31.659984 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:31.659940    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:31.759788 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:31.759754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:37.464532 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:37.464496    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"]
Jul 21 13:29:37.465083 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:37.464740    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq" podUID="269968da-d633-4046-8d8d-877fd30a93db" containerName="maas-api" containerID="cri-o://5fd066f9caf95860a0fb1321d46b549e4bf09d3a5a88ab508b7de011312723bf" gracePeriod=30
Jul 21 13:29:37.501357 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:37.501320    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:37.711099 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:37.711074    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:37.898520 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:37.898472    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269968da-d633-4046-8d8d-877fd30a93db-maas-api-tls\") pod \"269968da-d633-4046-8d8d-877fd30a93db\" (UID: \"269968da-d633-4046-8d8d-877fd30a93db\") "
Jul 21 13:29:37.898713 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:37.898543    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269968da-d633-4046-8d8d-877fd30a93db-openshift-service-ca\") pod \"269968da-d633-4046-8d8d-877fd30a93db\" (UID: \"269968da-d633-4046-8d8d-877fd30a93db\") "
Jul 21 13:29:37.898713 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:37.898641    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwbk5\" (UniqueName: \"kubernetes.io/projected/269968da-d633-4046-8d8d-877fd30a93db-kube-api-access-xwbk5\") pod \"269968da-d633-4046-8d8d-877fd30a93db\" (UID: \"269968da-d633-4046-8d8d-877fd30a93db\") "
Jul 21 13:29:37.898947 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:37.898914    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/269968da-d633-4046-8d8d-877fd30a93db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "269968da-d633-4046-8d8d-877fd30a93db" (UID: "269968da-d633-4046-8d8d-877fd30a93db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:29:37.900810 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:37.900788    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/269968da-d633-4046-8d8d-877fd30a93db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "269968da-d633-4046-8d8d-877fd30a93db" (UID: "269968da-d633-4046-8d8d-877fd30a93db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:29:37.900925 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:37.900900    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/269968da-d633-4046-8d8d-877fd30a93db-kube-api-access-xwbk5" (OuterVolumeSpecName: "kube-api-access-xwbk5") pod "269968da-d633-4046-8d8d-877fd30a93db" (UID: "269968da-d633-4046-8d8d-877fd30a93db"). InnerVolumeSpecName "kube-api-access-xwbk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:29:37.999461 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:37.999428    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwbk5\" (UniqueName: \"kubernetes.io/projected/269968da-d633-4046-8d8d-877fd30a93db-kube-api-access-xwbk5\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:29:37.999461 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:37.999453    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269968da-d633-4046-8d8d-877fd30a93db-maas-api-tls\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:29:37.999461 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:37.999462    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269968da-d633-4046-8d8d-877fd30a93db-openshift-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:29:38.037355 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:38.037316    2563 generic.go:356] "Generic (PLEG): container finished" podID="269968da-d633-4046-8d8d-877fd30a93db" containerID="5fd066f9caf95860a0fb1321d46b549e4bf09d3a5a88ab508b7de011312723bf" exitCode=0
Jul 21 13:29:38.037533 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:38.037367    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"
Jul 21 13:29:38.037533 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:38.037400    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq" event={"ID":"269968da-d633-4046-8d8d-877fd30a93db","Type":"ContainerDied","Data":"5fd066f9caf95860a0fb1321d46b549e4bf09d3a5a88ab508b7de011312723bf"}
Jul 21 13:29:38.037533 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:38.037451    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq" event={"ID":"269968da-d633-4046-8d8d-877fd30a93db","Type":"ContainerDied","Data":"eb796623600789f2c0414ab1aab858b4292589d830dae176adfcf08146953257"}
Jul 21 13:29:38.037533 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:38.037471    2563 scope.go:117] "RemoveContainer" containerID="5fd066f9caf95860a0fb1321d46b549e4bf09d3a5a88ab508b7de011312723bf"
Jul 21 13:29:38.045399 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:38.045376    2563 scope.go:117] "RemoveContainer" containerID="5fd066f9caf95860a0fb1321d46b549e4bf09d3a5a88ab508b7de011312723bf"
Jul 21 13:29:38.045657 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:29:38.045634    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fd066f9caf95860a0fb1321d46b549e4bf09d3a5a88ab508b7de011312723bf\": container with ID starting with 5fd066f9caf95860a0fb1321d46b549e4bf09d3a5a88ab508b7de011312723bf not found: ID does not exist" containerID="5fd066f9caf95860a0fb1321d46b549e4bf09d3a5a88ab508b7de011312723bf"
Jul 21 13:29:38.045734 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:38.045686    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fd066f9caf95860a0fb1321d46b549e4bf09d3a5a88ab508b7de011312723bf"} err="failed to get container status \"5fd066f9caf95860a0fb1321d46b549e4bf09d3a5a88ab508b7de011312723bf\": rpc error: code = NotFound desc = could not find container \"5fd066f9caf95860a0fb1321d46b549e4bf09d3a5a88ab508b7de011312723bf\": container with ID starting with 5fd066f9caf95860a0fb1321d46b549e4bf09d3a5a88ab508b7de011312723bf not found: ID does not exist"
Jul 21 13:29:38.056870 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:38.056843    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"]
Jul 21 13:29:38.062928 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:38.062906    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b6860c-7cbfd86797-698jq"]
Jul 21 13:29:39.995445 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:39.995409    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="269968da-d633-4046-8d8d-877fd30a93db" path="/var/lib/kubelet/pods/269968da-d633-4046-8d8d-877fd30a93db/volumes"
Jul 21 13:29:42.184460 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:42.184424    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:42.184904 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:42.184766    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9"]
Jul 21 13:29:42.187470 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:42.187445    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b6860c-dzhb9"]
Jul 21 13:29:42.571526 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:42.571440    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:43.995742 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:43.995709    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6072b55-c926-4afb-98b3-7991b47f7c1a" path="/var/lib/kubelet/pods/d6072b55-c926-4afb-98b3-7991b47f7c1a/volumes"
Jul 21 13:29:49.027634 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.027590    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:49.111959 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.111922    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:49.266485 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.266447    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:29:49.379973 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.379913    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk"]
Jul 21 13:29:49.380265 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.380252    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="269968da-d633-4046-8d8d-877fd30a93db" containerName="maas-api"
Jul 21 13:29:49.380337 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.380266    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="269968da-d633-4046-8d8d-877fd30a93db" containerName="maas-api"
Jul 21 13:29:49.380337 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.380274    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6072b55-c926-4afb-98b3-7991b47f7c1a" containerName="revoke-keys"
Jul 21 13:29:49.380337 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.380280    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6072b55-c926-4afb-98b3-7991b47f7c1a" containerName="revoke-keys"
Jul 21 13:29:49.380429 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.380343    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6072b55-c926-4afb-98b3-7991b47f7c1a" containerName="revoke-keys"
Jul 21 13:29:49.380429 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.380352    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="269968da-d633-4046-8d8d-877fd30a93db" containerName="maas-api"
Jul 21 13:29:49.383408 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.383390    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk"
Jul 21 13:29:49.386198 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.386178    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qhp7l\""
Jul 21 13:29:49.394060 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.394037    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk"]
Jul 21 13:29:49.395633 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.395610    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d6310d3-b3d1-4cc5-803e-d0fac10410ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk\" (UID: \"9d6310d3-b3d1-4cc5-803e-d0fac10410ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk"
Jul 21 13:29:49.496970 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.496929    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d6310d3-b3d1-4cc5-803e-d0fac10410ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk\" (UID: \"9d6310d3-b3d1-4cc5-803e-d0fac10410ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk"
Jul 21 13:29:49.497579 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.497560    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d6310d3-b3d1-4cc5-803e-d0fac10410ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk\" (UID: \"9d6310d3-b3d1-4cc5-803e-d0fac10410ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk"
Jul 21 13:29:49.693780 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.693694    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk"
Jul 21 13:29:49.833697 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:49.833670    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk"]
Jul 21 13:29:49.834902 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:29:49.834877    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d6310d3_b3d1_4cc5_803e_d0fac10410ae.slice/crio-ff2763f4ef02652020d71ac04665b9d72fe6de0a599b6c7d091cedf2810c8e79 WatchSource:0}: Error finding container ff2763f4ef02652020d71ac04665b9d72fe6de0a599b6c7d091cedf2810c8e79: Status 404 returned error can't find the container with id ff2763f4ef02652020d71ac04665b9d72fe6de0a599b6c7d091cedf2810c8e79
Jul 21 13:29:50.091381 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:50.091346    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk" event={"ID":"9d6310d3-b3d1-4cc5-803e-d0fac10410ae","Type":"ContainerStarted","Data":"2fc9482dc766231e0d7e4951556e2db0d2b719478d5ad1011b9f990ad703c2fe"}
Jul 21 13:29:50.091733 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:50.091416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk" event={"ID":"9d6310d3-b3d1-4cc5-803e-d0fac10410ae","Type":"ContainerStarted","Data":"ff2763f4ef02652020d71ac04665b9d72fe6de0a599b6c7d091cedf2810c8e79"}
Jul 21 13:29:50.112344 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:50.112265    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk" podStartSLOduration=1.112247598 podStartE2EDuration="1.112247598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:29:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:29:50.111982133 +0000 UTC m=+2322.637485305" watchObservedRunningTime="2026-07-21 13:29:50.112247598 +0000 UTC m=+2322.637750756"
Jul 21 13:29:51.098277 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:51.098241    2563 generic.go:356] "Generic (PLEG): container finished" podID="9d6310d3-b3d1-4cc5-803e-d0fac10410ae" containerID="2fc9482dc766231e0d7e4951556e2db0d2b719478d5ad1011b9f990ad703c2fe" exitCode=0
Jul 21 13:29:51.098711 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:51.098311    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk" event={"ID":"9d6310d3-b3d1-4cc5-803e-d0fac10410ae","Type":"ContainerDied","Data":"2fc9482dc766231e0d7e4951556e2db0d2b719478d5ad1011b9f990ad703c2fe"}
Jul 21 13:29:52.235519 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:52.235496    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk"
Jul 21 13:29:52.323437 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:52.323400    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d6310d3-b3d1-4cc5-803e-d0fac10410ae-maas-api-service-ca\") pod \"9d6310d3-b3d1-4cc5-803e-d0fac10410ae\" (UID: \"9d6310d3-b3d1-4cc5-803e-d0fac10410ae\") "
Jul 21 13:29:52.323774 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:52.323749    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d6310d3-b3d1-4cc5-803e-d0fac10410ae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9d6310d3-b3d1-4cc5-803e-d0fac10410ae" (UID: "9d6310d3-b3d1-4cc5-803e-d0fac10410ae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:29:52.424330 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:52.424237    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d6310d3-b3d1-4cc5-803e-d0fac10410ae-maas-api-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:29:53.108221 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:53.108189    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk"
Jul 21 13:29:53.108423 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:53.108232    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk" event={"ID":"9d6310d3-b3d1-4cc5-803e-d0fac10410ae","Type":"ContainerDied","Data":"ff2763f4ef02652020d71ac04665b9d72fe6de0a599b6c7d091cedf2810c8e79"}
Jul 21 13:29:53.108423 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:29:53.108280    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff2763f4ef02652020d71ac04665b9d72fe6de0a599b6c7d091cedf2810c8e79"
Jul 21 13:30:00.147521 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.147439    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp"]
Jul 21 13:30:00.147922 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.147759    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6310d3-b3d1-4cc5-803e-d0fac10410ae" containerName="revoke-keys"
Jul 21 13:30:00.147922 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.147771    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6310d3-b3d1-4cc5-803e-d0fac10410ae" containerName="revoke-keys"
Jul 21 13:30:00.147922 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.147839    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d6310d3-b3d1-4cc5-803e-d0fac10410ae" containerName="revoke-keys"
Jul 21 13:30:00.151152 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.151129    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp"
Jul 21 13:30:00.154327 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.154284    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qhp7l\""
Jul 21 13:30:00.155382 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.155352    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744010-njbtm"]
Jul 21 13:30:00.158540 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.158524    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744010-njbtm"
Jul 21 13:30:00.161648 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.161627    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp"
Jul 21 13:30:00.163121 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.163091    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp"]
Jul 21 13:30:00.167816 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.167791    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744010-njbtm"]
Jul 21 13:30:00.173923 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.173899    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744010-njbtm"
Jul 21 13:30:00.345492 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.345351    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744010-njbtm"]
Jul 21 13:30:00.348159 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:30:00.348130    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e56196_c194_4b9f_bbea_57badedf0e47.slice/crio-8963832e848fd32514ac6ad67cc4f27d1325cf2976ad8097ff8aeddb662c63ce WatchSource:0}: Error finding container 8963832e848fd32514ac6ad67cc4f27d1325cf2976ad8097ff8aeddb662c63ce: Status 404 returned error can't find the container with id 8963832e848fd32514ac6ad67cc4f27d1325cf2976ad8097ff8aeddb662c63ce
Jul 21 13:30:00.529472 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:00.529438    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp"]
Jul 21 13:30:00.530756 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:30:00.530727    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe0a8880_3c32_4185_9167_3ac8bebe553b.slice/crio-1a9404d26a442bd58ae66f1b5358829cb53fe67a3853308ab571278ac29d41b4 WatchSource:0}: Error finding container 1a9404d26a442bd58ae66f1b5358829cb53fe67a3853308ab571278ac29d41b4: Status 404 returned error can't find the container with id 1a9404d26a442bd58ae66f1b5358829cb53fe67a3853308ab571278ac29d41b4
Jul 21 13:30:01.155117 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:01.155078    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp" event={"ID":"be0a8880-3c32-4185-9167-3ac8bebe553b","Type":"ContainerStarted","Data":"e3ef2371c00c314c2838fcf93fdef17bf42ef2b908768d62e67ad18699fc9252"}
Jul 21 13:30:01.155570 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:01.155140    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp" event={"ID":"be0a8880-3c32-4185-9167-3ac8bebe553b","Type":"ContainerStarted","Data":"1a9404d26a442bd58ae66f1b5358829cb53fe67a3853308ab571278ac29d41b4"}
Jul 21 13:30:01.156805 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:01.156779    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744010-njbtm" event={"ID":"36e56196-c194-4b9f-bbea-57badedf0e47","Type":"ContainerStarted","Data":"31d57735898b6e58e3a80236d3af9f66c5824e8ba6be758cf4b97c244575008d"}
Jul 21 13:30:01.156931 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:01.156827    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744010-njbtm" event={"ID":"36e56196-c194-4b9f-bbea-57badedf0e47","Type":"ContainerStarted","Data":"8963832e848fd32514ac6ad67cc4f27d1325cf2976ad8097ff8aeddb662c63ce"}
Jul 21 13:30:01.176829 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:01.176777    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp" podStartSLOduration=1.176759266 podStartE2EDuration="1.176759266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:30:01.175577337 +0000 UTC m=+2333.701080497" watchObservedRunningTime="2026-07-21 13:30:01.176759266 +0000 UTC m=+2333.702262420"
Jul 21 13:30:01.193358 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:01.193283    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744010-njbtm" podStartSLOduration=1.193267648 podStartE2EDuration="1.193267648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:30:01.191620236 +0000 UTC m=+2333.717123392" watchObservedRunningTime="2026-07-21 13:30:01.193267648 +0000 UTC m=+2333.718770803"
Jul 21 13:30:02.162512 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:02.162478    2563 generic.go:356] "Generic (PLEG): container finished" podID="36e56196-c194-4b9f-bbea-57badedf0e47" containerID="31d57735898b6e58e3a80236d3af9f66c5824e8ba6be758cf4b97c244575008d" exitCode=0
Jul 21 13:30:02.162965 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:02.162569    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744010-njbtm" event={"ID":"36e56196-c194-4b9f-bbea-57badedf0e47","Type":"ContainerDied","Data":"31d57735898b6e58e3a80236d3af9f66c5824e8ba6be758cf4b97c244575008d"}
Jul 21 13:30:02.165175 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:02.165135    2563 generic.go:356] "Generic (PLEG): container finished" podID="be0a8880-3c32-4185-9167-3ac8bebe553b" containerID="e3ef2371c00c314c2838fcf93fdef17bf42ef2b908768d62e67ad18699fc9252" exitCode=0
Jul 21 13:30:02.165351 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:02.165241    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp" event={"ID":"be0a8880-3c32-4185-9167-3ac8bebe553b","Type":"ContainerDied","Data":"e3ef2371c00c314c2838fcf93fdef17bf42ef2b908768d62e67ad18699fc9252"}
Jul 21 13:30:03.313514 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:03.313490    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744010-njbtm"
Jul 21 13:30:03.316609 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:03.316586    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp"
Jul 21 13:30:04.175751 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:04.175722    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp"
Jul 21 13:30:04.175935 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:04.175767    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp" event={"ID":"be0a8880-3c32-4185-9167-3ac8bebe553b","Type":"ContainerDied","Data":"1a9404d26a442bd58ae66f1b5358829cb53fe67a3853308ab571278ac29d41b4"}
Jul 21 13:30:04.175935 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:04.175814    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a9404d26a442bd58ae66f1b5358829cb53fe67a3853308ab571278ac29d41b4"
Jul 21 13:30:04.177003 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:04.176969    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744010-njbtm" event={"ID":"36e56196-c194-4b9f-bbea-57badedf0e47","Type":"ContainerDied","Data":"8963832e848fd32514ac6ad67cc4f27d1325cf2976ad8097ff8aeddb662c63ce"}
Jul 21 13:30:04.177137 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:04.177018    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8963832e848fd32514ac6ad67cc4f27d1325cf2976ad8097ff8aeddb662c63ce"
Jul 21 13:30:04.177137 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:04.177033    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744010-njbtm"
Jul 21 13:30:04.944135 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:04.944099    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"]
Jul 21 13:30:04.944925 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:04.944870    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj" podUID="abdac7d4-0bbf-483c-a964-3bd1abaa1920" containerName="maas-api" containerID="cri-o://cdd4573a68020ca11b282f5d22dfb3ce1eb729985cad98b103a83e695e73e632" gracePeriod=30
Jul 21 13:30:04.972031 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:04.971996    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:04.976599 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:04.976565    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp"]
Jul 21 13:30:04.980598 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:04.980568    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f114ec-29744010-dv5qp"]
Jul 21 13:30:05.184765 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:05.184736    2563 generic.go:356] "Generic (PLEG): container finished" podID="abdac7d4-0bbf-483c-a964-3bd1abaa1920" containerID="cdd4573a68020ca11b282f5d22dfb3ce1eb729985cad98b103a83e695e73e632" exitCode=0
Jul 21 13:30:05.184887 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:05.184806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj" event={"ID":"abdac7d4-0bbf-483c-a964-3bd1abaa1920","Type":"ContainerDied","Data":"cdd4573a68020ca11b282f5d22dfb3ce1eb729985cad98b103a83e695e73e632"}
Jul 21 13:30:05.199507 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:05.199437    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:30:05.343552 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:05.343512    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abdac7d4-0bbf-483c-a964-3bd1abaa1920-openshift-service-ca\") pod \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\" (UID: \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\") "
Jul 21 13:30:05.343736 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:05.343634    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjlcl\" (UniqueName: \"kubernetes.io/projected/abdac7d4-0bbf-483c-a964-3bd1abaa1920-kube-api-access-rjlcl\") pod \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\" (UID: \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\") "
Jul 21 13:30:05.343736 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:05.343660    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abdac7d4-0bbf-483c-a964-3bd1abaa1920-maas-api-tls\") pod \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\" (UID: \"abdac7d4-0bbf-483c-a964-3bd1abaa1920\") "
Jul 21 13:30:05.343983 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:05.343956    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abdac7d4-0bbf-483c-a964-3bd1abaa1920-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "abdac7d4-0bbf-483c-a964-3bd1abaa1920" (UID: "abdac7d4-0bbf-483c-a964-3bd1abaa1920"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:30:05.345878 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:05.345848    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abdac7d4-0bbf-483c-a964-3bd1abaa1920-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "abdac7d4-0bbf-483c-a964-3bd1abaa1920" (UID: "abdac7d4-0bbf-483c-a964-3bd1abaa1920"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:30:05.345994 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:05.345928    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abdac7d4-0bbf-483c-a964-3bd1abaa1920-kube-api-access-rjlcl" (OuterVolumeSpecName: "kube-api-access-rjlcl") pod "abdac7d4-0bbf-483c-a964-3bd1abaa1920" (UID: "abdac7d4-0bbf-483c-a964-3bd1abaa1920"). InnerVolumeSpecName "kube-api-access-rjlcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:30:05.444537 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:05.444483    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjlcl\" (UniqueName: \"kubernetes.io/projected/abdac7d4-0bbf-483c-a964-3bd1abaa1920-kube-api-access-rjlcl\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:30:05.444537 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:05.444532    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abdac7d4-0bbf-483c-a964-3bd1abaa1920-maas-api-tls\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:30:05.444537 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:05.444542    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abdac7d4-0bbf-483c-a964-3bd1abaa1920-openshift-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:30:05.995252 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:05.995207    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be0a8880-3c32-4185-9167-3ac8bebe553b" path="/var/lib/kubelet/pods/be0a8880-3c32-4185-9167-3ac8bebe553b/volumes"
Jul 21 13:30:06.191354 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:06.191327    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"
Jul 21 13:30:06.191354 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:06.191342    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj" event={"ID":"abdac7d4-0bbf-483c-a964-3bd1abaa1920","Type":"ContainerDied","Data":"67fa2a072d91f749b0d2b672329c1f22f3f3c492ebcab5cd6e30c4af6c09c556"}
Jul 21 13:30:06.191630 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:06.191401    2563 scope.go:117] "RemoveContainer" containerID="cdd4573a68020ca11b282f5d22dfb3ce1eb729985cad98b103a83e695e73e632"
Jul 21 13:30:06.210920 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:06.210887    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"]
Jul 21 13:30:06.216101 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:06.216072    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f114ec-664d6b58b9-tl4gj"]
Jul 21 13:30:07.996069 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:07.996031    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abdac7d4-0bbf-483c-a964-3bd1abaa1920" path="/var/lib/kubelet/pods/abdac7d4-0bbf-483c-a964-3bd1abaa1920/volumes"
Jul 21 13:30:08.465764 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:08.465727    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:08.635370 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:08.635310    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:09.472225 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:09.472186    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk"]
Jul 21 13:30:09.476185 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:09.476159    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f114ec-7t5mk"]
Jul 21 13:30:09.478707 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:09.478679    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:09.995996 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:09.995960    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d6310d3-b3d1-4cc5-803e-d0fac10410ae" path="/var/lib/kubelet/pods/9d6310d3-b3d1-4cc5-803e-d0fac10410ae/volumes"
Jul 21 13:30:11.487752 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:11.487720    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:16.031048 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:16.030968    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:16.129576 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:16.129542    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:16.314287 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:16.314192    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:16.558875 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:16.558843    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:16.912055 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:16.912017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:17.283083 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:17.283000    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:17.732417 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:17.732383    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:20.174255 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.174224    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:20.277307 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.277267    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"]
Jul 21 13:30:20.277652 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.277634    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be0a8880-3c32-4185-9167-3ac8bebe553b" containerName="cleanup"
Jul 21 13:30:20.277735 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.277656    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="be0a8880-3c32-4185-9167-3ac8bebe553b" containerName="cleanup"
Jul 21 13:30:20.277735 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.277677    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e56196-c194-4b9f-bbea-57badedf0e47" containerName="cleanup"
Jul 21 13:30:20.277735 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.277686    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e56196-c194-4b9f-bbea-57badedf0e47" containerName="cleanup"
Jul 21 13:30:20.277735 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.277705    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abdac7d4-0bbf-483c-a964-3bd1abaa1920" containerName="maas-api"
Jul 21 13:30:20.277735 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.277714    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="abdac7d4-0bbf-483c-a964-3bd1abaa1920" containerName="maas-api"
Jul 21 13:30:20.277992 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.277797    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="abdac7d4-0bbf-483c-a964-3bd1abaa1920" containerName="maas-api"
Jul 21 13:30:20.277992 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.277810    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="be0a8880-3c32-4185-9167-3ac8bebe553b" containerName="cleanup"
Jul 21 13:30:20.277992 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.277819    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e56196-c194-4b9f-bbea-57badedf0e47" containerName="cleanup"
Jul 21 13:30:20.280448 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.280422    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:20.280926 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.280907    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:20.283711 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.283692    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-3af280-serving-cert\""
Jul 21 13:30:20.283825 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.283725    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 13:30:20.283825 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.283726    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pkltq\""
Jul 21 13:30:20.285539 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.285519    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 13:30:20.292313 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.292270    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"]
Jul 21 13:30:20.375122 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.375076    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf46e60a-8695-4199-8879-36109c6d2d5c-maas-api-tls\") pod \"maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf\" (UID: \"bf46e60a-8695-4199-8879-36109c6d2d5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:20.375347 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.375319    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krg9c\" (UniqueName: \"kubernetes.io/projected/bf46e60a-8695-4199-8879-36109c6d2d5c-kube-api-access-krg9c\") pod \"maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf\" (UID: \"bf46e60a-8695-4199-8879-36109c6d2d5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:20.375428 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.375383    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf46e60a-8695-4199-8879-36109c6d2d5c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf\" (UID: \"bf46e60a-8695-4199-8879-36109c6d2d5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:20.377841 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.377812    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:20.476250 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.476147    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf46e60a-8695-4199-8879-36109c6d2d5c-maas-api-tls\") pod \"maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf\" (UID: \"bf46e60a-8695-4199-8879-36109c6d2d5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:20.476250 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.476217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krg9c\" (UniqueName: \"kubernetes.io/projected/bf46e60a-8695-4199-8879-36109c6d2d5c-kube-api-access-krg9c\") pod \"maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf\" (UID: \"bf46e60a-8695-4199-8879-36109c6d2d5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:20.476250 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.476249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf46e60a-8695-4199-8879-36109c6d2d5c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf\" (UID: \"bf46e60a-8695-4199-8879-36109c6d2d5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:20.476579 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:30:20.476346    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-serving-cert: secret "maas-api-e2e-shared-a-3af280-serving-cert" not found
Jul 21 13:30:20.476579 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:30:20.476437    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf46e60a-8695-4199-8879-36109c6d2d5c-maas-api-tls podName:bf46e60a-8695-4199-8879-36109c6d2d5c nodeName:}" failed. No retries permitted until 2026-07-21 13:30:20.976416653 +0000 UTC m=+2353.501919789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bf46e60a-8695-4199-8879-36109c6d2d5c-maas-api-tls") pod "maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf" (UID: "bf46e60a-8695-4199-8879-36109c6d2d5c") : secret "maas-api-e2e-shared-a-3af280-serving-cert" not found
Jul 21 13:30:20.476908 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.476889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf46e60a-8695-4199-8879-36109c6d2d5c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf\" (UID: \"bf46e60a-8695-4199-8879-36109c6d2d5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:20.488167 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.488132    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krg9c\" (UniqueName: \"kubernetes.io/projected/bf46e60a-8695-4199-8879-36109c6d2d5c-kube-api-access-krg9c\") pod \"maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf\" (UID: \"bf46e60a-8695-4199-8879-36109c6d2d5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:20.715166 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.715131    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:20.871628 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.871589    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:20.974699 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.974666    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:20.981874 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.981851    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf46e60a-8695-4199-8879-36109c6d2d5c-maas-api-tls\") pod \"maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf\" (UID: \"bf46e60a-8695-4199-8879-36109c6d2d5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:20.984390 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:20.984368    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf46e60a-8695-4199-8879-36109c6d2d5c-maas-api-tls\") pod \"maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf\" (UID: \"bf46e60a-8695-4199-8879-36109c6d2d5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:21.190653 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:21.190561    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:21.334865 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:21.334841    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"]
Jul 21 13:30:21.336661 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:30:21.336632    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf46e60a_8695_4199_8879_36109c6d2d5c.slice/crio-5630155f7079bdc50cd66092e1a86793d9de2710fff35c69a30c93e5e71b8a57 WatchSource:0}: Error finding container 5630155f7079bdc50cd66092e1a86793d9de2710fff35c69a30c93e5e71b8a57: Status 404 returned error can't find the container with id 5630155f7079bdc50cd66092e1a86793d9de2710fff35c69a30c93e5e71b8a57
Jul 21 13:30:22.269899 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:22.269861    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf" event={"ID":"bf46e60a-8695-4199-8879-36109c6d2d5c","Type":"ContainerStarted","Data":"6467d38d655db69605c77109e5fc6362bb1eba306e20621ff6b6afa19e8ab570"}
Jul 21 13:30:22.270326 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:22.269923    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf" event={"ID":"bf46e60a-8695-4199-8879-36109c6d2d5c","Type":"ContainerStarted","Data":"5630155f7079bdc50cd66092e1a86793d9de2710fff35c69a30c93e5e71b8a57"}
Jul 21 13:30:22.270326 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:22.269965    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:22.291051 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:22.290995    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf" podStartSLOduration=1.805766548 podStartE2EDuration="2.290976635s" totalImagesPullingTime="485.210087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:30:22.289693027 +0000 UTC m=+2354.815196186" watchObservedRunningTime="2026-07-21 13:30:22.290976635 +0000 UTC m=+2354.816479790"
Jul 21 13:30:24.654283 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:24.654248    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:24.977975 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:24.977887    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:25.332923 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:25.332834    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:25.476667 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:25.476634    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:28.206242 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.206213    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:28.282219 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.282084    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:30:28.289812 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.289784    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"]
Jul 21 13:30:28.293165 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.293144    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:28.296099 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.296074    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-2df168-serving-cert\""
Jul 21 13:30:28.299023 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.298964    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:28.304816 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.304779    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"]
Jul 21 13:30:28.395634 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.395601    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:28.452159 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.452123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc52n\" (UniqueName: \"kubernetes.io/projected/d60f55ed-bc62-47dc-9312-53c0da604364-kube-api-access-bc52n\") pod \"maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:28.452387 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.452268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d60f55ed-bc62-47dc-9312-53c0da604364-maas-api-tls\") pod \"maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:28.452387 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.452334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d60f55ed-bc62-47dc-9312-53c0da604364-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:28.553038 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.552954    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d60f55ed-bc62-47dc-9312-53c0da604364-maas-api-tls\") pod \"maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:28.553038 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.552987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d60f55ed-bc62-47dc-9312-53c0da604364-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:28.553038 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.553037    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc52n\" (UniqueName: \"kubernetes.io/projected/d60f55ed-bc62-47dc-9312-53c0da604364-kube-api-access-bc52n\") pod \"maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:28.553271 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:30:28.553099    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-serving-cert: secret "maas-api-e2e-shared-b-2df168-serving-cert" not found
Jul 21 13:30:28.553271 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:30:28.553155    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d60f55ed-bc62-47dc-9312-53c0da604364-maas-api-tls podName:d60f55ed-bc62-47dc-9312-53c0da604364 nodeName:}" failed. No retries permitted until 2026-07-21 13:30:29.053139989 +0000 UTC m=+2361.578643123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d60f55ed-bc62-47dc-9312-53c0da604364-maas-api-tls") pod "maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck" (UID: "d60f55ed-bc62-47dc-9312-53c0da604364") : secret "maas-api-e2e-shared-b-2df168-serving-cert" not found
Jul 21 13:30:28.553747 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.553731    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d60f55ed-bc62-47dc-9312-53c0da604364-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:28.562602 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:28.562572    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc52n\" (UniqueName: \"kubernetes.io/projected/d60f55ed-bc62-47dc-9312-53c0da604364-kube-api-access-bc52n\") pod \"maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:29.003613 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:29.003577    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:29.057497 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:29.057456    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d60f55ed-bc62-47dc-9312-53c0da604364-maas-api-tls\") pod \"maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:29.057712 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:30:29.057659    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-serving-cert: secret "maas-api-e2e-shared-b-2df168-serving-cert" not found
Jul 21 13:30:29.057787 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:30:29.057761    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d60f55ed-bc62-47dc-9312-53c0da604364-maas-api-tls podName:d60f55ed-bc62-47dc-9312-53c0da604364 nodeName:}" failed. No retries permitted until 2026-07-21 13:30:30.057738907 +0000 UTC m=+2362.583242111 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d60f55ed-bc62-47dc-9312-53c0da604364-maas-api-tls") pod "maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck" (UID: "d60f55ed-bc62-47dc-9312-53c0da604364") : secret "maas-api-e2e-shared-b-2df168-serving-cert" not found
Jul 21 13:30:29.104985 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:29.104947    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:29.204141 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:29.204100    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:30.067315 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:30.067259    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d60f55ed-bc62-47dc-9312-53c0da604364-maas-api-tls\") pod \"maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:30.069826 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:30.069805    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d60f55ed-bc62-47dc-9312-53c0da604364-maas-api-tls\") pod \"maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:30.106888 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:30.106858    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:30.248044 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:30.248016    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"]
Jul 21 13:30:30.250657 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:30:30.250629    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd60f55ed_bc62_47dc_9312_53c0da604364.slice/crio-2657e69fd7427f2d83a75c3d43b40ecac50333f12188b0721ea7490d68dc6965 WatchSource:0}: Error finding container 2657e69fd7427f2d83a75c3d43b40ecac50333f12188b0721ea7490d68dc6965: Status 404 returned error can't find the container with id 2657e69fd7427f2d83a75c3d43b40ecac50333f12188b0721ea7490d68dc6965
Jul 21 13:30:30.310451 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:30.310409    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck" event={"ID":"d60f55ed-bc62-47dc-9312-53c0da604364","Type":"ContainerStarted","Data":"2657e69fd7427f2d83a75c3d43b40ecac50333f12188b0721ea7490d68dc6965"}
Jul 21 13:30:31.316318 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:31.316252    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck" event={"ID":"d60f55ed-bc62-47dc-9312-53c0da604364","Type":"ContainerStarted","Data":"75774dccd640bfc0fae702181011a6e7e62d15458aeb261e01e33e3e645b96df"}
Jul 21 13:30:31.316742 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:31.316343    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:31.341074 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:31.341022    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck" podStartSLOduration=2.9189107439999997 podStartE2EDuration="3.341005201s" totalImagesPullingTime="422.094457ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:30:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:30:31.339373967 +0000 UTC m=+2363.864877123" watchObservedRunningTime="2026-07-21 13:30:31.341005201 +0000 UTC m=+2363.866508379"
Jul 21 13:30:32.684239 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:32.684208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:37.327575 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:37.327538    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:30:41.738555 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:41.738507    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:41.849650 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:41.849613    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:51.218014 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:51.217979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:51.316553 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:51.316515    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:30:52.060567 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:30:52.060534    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:31:08.095963 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:31:08.095931    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:31:08.102796 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:31:08.102773    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:31:08.228390 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:31:08.228358    2563 scope.go:117] "RemoveContainer" containerID="891af4f67122c90962ca18f63463f339f6f1c54f9b319f0a9bf32bfe51d51bc9"
Jul 21 13:31:15.373859 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:31:15.373823    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:32:09.757416 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:32:09.757381    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:32:09.884032 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:32:09.884002    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:32:40.331365 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:32:40.331330    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:33:25.179362 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:33:25.179266    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:33:25.253645 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:33:25.253607    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:33:29.967945 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:33:29.967911    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:33:51.765101 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:33:51.765058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:33:51.831565 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:33:51.831524    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:33:53.784458 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:33:53.784419    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:34:23.287928 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:34:23.287891    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:34:30.036555 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:34:30.036484    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:34:31.397415 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:34:31.397378    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:34:45.646169 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:34:45.646126    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:35:08.265514 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:35:08.265481    2563 scope.go:117] "RemoveContainer" containerID="9d773a623851769574265c1d9742595ac6ee2cf39fb03de265bca0a08681490c"
Jul 21 13:35:09.365039 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:35:09.365001    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:35:09.464340 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:35:09.464310    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:35:18.557609 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:35:18.557572    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:35:38.577826 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:35:38.577775    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:35:38.677093 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:35:38.677044    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:35:42.731546 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:35:42.731499    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:35:42.936371 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:35:42.936334    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:35:52.070069 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:35:52.070034    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:35:52.169668 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:35:52.169629    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:35:59.992040 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:35:59.991964    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:08.121629 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:08.121591    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:36:08.129480 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:08.129455    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9674_cc4dd5b9-c37f-4f31-9ce0-0386f9cf478b/ovn-acl-logging/0.log"
Jul 21 13:36:08.285814 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:08.285780    2563 scope.go:117] "RemoveContainer" containerID="2fc9482dc766231e0d7e4951556e2db0d2b719478d5ad1011b9f990ad703c2fe"
Jul 21 13:36:08.293900 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:08.293878    2563 scope.go:117] "RemoveContainer" containerID="234b76a3b600028cf488a94b428009553a2aaf2af93713b7362fece9854a048b"
Jul 21 13:36:08.301701 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:08.301679    2563 scope.go:117] "RemoveContainer" containerID="e3ef2371c00c314c2838fcf93fdef17bf42ef2b908768d62e67ad18699fc9252"
Jul 21 13:36:13.065758 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:13.065699    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:14.034914 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:14.034881    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:33.282582 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:33.282548    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:33.382672 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:33.382630    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:43.163455 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.163418    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"]
Jul 21 13:36:43.163967 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.163711    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf" podUID="bf46e60a-8695-4199-8879-36109c6d2d5c" containerName="maas-api" containerID="cri-o://6467d38d655db69605c77109e5fc6362bb1eba306e20621ff6b6afa19e8ab570" gracePeriod=30
Jul 21 13:36:43.184596 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.184555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:43.276199 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.276158    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf" podUID="bf46e60a-8695-4199-8879-36109c6d2d5c" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.51:8443/health\": dial tcp 10.133.0.51:8443: connect: connection refused"
Jul 21 13:36:43.422574 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.422506    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:36:43.499472 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.499432    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf46e60a-8695-4199-8879-36109c6d2d5c-openshift-service-ca\") pod \"bf46e60a-8695-4199-8879-36109c6d2d5c\" (UID: \"bf46e60a-8695-4199-8879-36109c6d2d5c\") "
Jul 21 13:36:43.499663 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.499509    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krg9c\" (UniqueName: \"kubernetes.io/projected/bf46e60a-8695-4199-8879-36109c6d2d5c-kube-api-access-krg9c\") pod \"bf46e60a-8695-4199-8879-36109c6d2d5c\" (UID: \"bf46e60a-8695-4199-8879-36109c6d2d5c\") "
Jul 21 13:36:43.499663 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.499539    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf46e60a-8695-4199-8879-36109c6d2d5c-maas-api-tls\") pod \"bf46e60a-8695-4199-8879-36109c6d2d5c\" (UID: \"bf46e60a-8695-4199-8879-36109c6d2d5c\") "
Jul 21 13:36:43.499889 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.499865    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf46e60a-8695-4199-8879-36109c6d2d5c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bf46e60a-8695-4199-8879-36109c6d2d5c" (UID: "bf46e60a-8695-4199-8879-36109c6d2d5c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:36:43.501787 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.501757    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf46e60a-8695-4199-8879-36109c6d2d5c-kube-api-access-krg9c" (OuterVolumeSpecName: "kube-api-access-krg9c") pod "bf46e60a-8695-4199-8879-36109c6d2d5c" (UID: "bf46e60a-8695-4199-8879-36109c6d2d5c"). InnerVolumeSpecName "kube-api-access-krg9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:36:43.501916 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.501892    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf46e60a-8695-4199-8879-36109c6d2d5c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bf46e60a-8695-4199-8879-36109c6d2d5c" (UID: "bf46e60a-8695-4199-8879-36109c6d2d5c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:36:43.600350 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.600268    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf46e60a-8695-4199-8879-36109c6d2d5c-openshift-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:36:43.600350 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.600341    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krg9c\" (UniqueName: \"kubernetes.io/projected/bf46e60a-8695-4199-8879-36109c6d2d5c-kube-api-access-krg9c\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:36:43.600350 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:43.600353    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf46e60a-8695-4199-8879-36109c6d2d5c-maas-api-tls\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:36:44.018964 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:44.018931    2563 generic.go:356] "Generic (PLEG): container finished" podID="bf46e60a-8695-4199-8879-36109c6d2d5c" containerID="6467d38d655db69605c77109e5fc6362bb1eba306e20621ff6b6afa19e8ab570" exitCode=0
Jul 21 13:36:44.019131 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:44.018976    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"
Jul 21 13:36:44.019131 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:44.019018    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf" event={"ID":"bf46e60a-8695-4199-8879-36109c6d2d5c","Type":"ContainerDied","Data":"6467d38d655db69605c77109e5fc6362bb1eba306e20621ff6b6afa19e8ab570"}
Jul 21 13:36:44.019131 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:44.019069    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf" event={"ID":"bf46e60a-8695-4199-8879-36109c6d2d5c","Type":"ContainerDied","Data":"5630155f7079bdc50cd66092e1a86793d9de2710fff35c69a30c93e5e71b8a57"}
Jul 21 13:36:44.019131 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:44.019094    2563 scope.go:117] "RemoveContainer" containerID="6467d38d655db69605c77109e5fc6362bb1eba306e20621ff6b6afa19e8ab570"
Jul 21 13:36:44.027880 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:44.027858    2563 scope.go:117] "RemoveContainer" containerID="6467d38d655db69605c77109e5fc6362bb1eba306e20621ff6b6afa19e8ab570"
Jul 21 13:36:44.028166 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:36:44.028144    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6467d38d655db69605c77109e5fc6362bb1eba306e20621ff6b6afa19e8ab570\": container with ID starting with 6467d38d655db69605c77109e5fc6362bb1eba306e20621ff6b6afa19e8ab570 not found: ID does not exist" containerID="6467d38d655db69605c77109e5fc6362bb1eba306e20621ff6b6afa19e8ab570"
Jul 21 13:36:44.028251 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:44.028177    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6467d38d655db69605c77109e5fc6362bb1eba306e20621ff6b6afa19e8ab570"} err="failed to get container status \"6467d38d655db69605c77109e5fc6362bb1eba306e20621ff6b6afa19e8ab570\": rpc error: code = NotFound desc = could not find container \"6467d38d655db69605c77109e5fc6362bb1eba306e20621ff6b6afa19e8ab570\": container with ID starting with 6467d38d655db69605c77109e5fc6362bb1eba306e20621ff6b6afa19e8ab570 not found: ID does not exist"
Jul 21 13:36:44.038134 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:44.038101    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"]
Jul 21 13:36:44.042628 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:44.042599    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3af280-7f9cff5c79-nlskf"]
Jul 21 13:36:45.995522 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:45.995486    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf46e60a-8695-4199-8879-36109c6d2d5c" path="/var/lib/kubelet/pods/bf46e60a-8695-4199-8879-36109c6d2d5c/volumes"
Jul 21 13:36:47.830812 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:47.830773    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:48.236038 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:48.236003    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:54.564065 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:54.564024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:54.642792 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:54.642761    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:54.842478 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:54.842388    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:55.071490 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.071443    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:55.167089 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.167058    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c"]
Jul 21 13:36:55.167426 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.167413    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf46e60a-8695-4199-8879-36109c6d2d5c" containerName="maas-api"
Jul 21 13:36:55.167477 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.167428    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf46e60a-8695-4199-8879-36109c6d2d5c" containerName="maas-api"
Jul 21 13:36:55.167512 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.167495    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf46e60a-8695-4199-8879-36109c6d2d5c" containerName="maas-api"
Jul 21 13:36:55.170704 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.170687    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c"
Jul 21 13:36:55.173582 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.173566    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qhp7l\""
Jul 21 13:36:55.179462 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.179373    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c"]
Jul 21 13:36:55.303312 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.303219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3e716a-13dc-464e-8be7-c7503dc1bce6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c\" (UID: \"ff3e716a-13dc-464e-8be7-c7503dc1bce6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c"
Jul 21 13:36:55.404143 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.404099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3e716a-13dc-464e-8be7-c7503dc1bce6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c\" (UID: \"ff3e716a-13dc-464e-8be7-c7503dc1bce6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c"
Jul 21 13:36:55.404786 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.404766    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3e716a-13dc-464e-8be7-c7503dc1bce6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c\" (UID: \"ff3e716a-13dc-464e-8be7-c7503dc1bce6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c"
Jul 21 13:36:55.483438 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.483351    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c"
Jul 21 13:36:55.624163 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.624133    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c"]
Jul 21 13:36:55.626168 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:36:55.626143    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3e716a_13dc_464e_8be7_c7503dc1bce6.slice/crio-53e3339b2cf03f49b14ce27a778449723f72449578886dd3b6a9d8b2873e6f35 WatchSource:0}: Error finding container 53e3339b2cf03f49b14ce27a778449723f72449578886dd3b6a9d8b2873e6f35: Status 404 returned error can't find the container with id 53e3339b2cf03f49b14ce27a778449723f72449578886dd3b6a9d8b2873e6f35
Jul 21 13:36:55.768849 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.768759    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:55.871325 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:55.871280    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:36:56.075428 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:56.075323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c" event={"ID":"ff3e716a-13dc-464e-8be7-c7503dc1bce6","Type":"ContainerStarted","Data":"c28e5da0c6302b630c41832adc0d065554b585fbde1e99e90e898d5649d896d3"}
Jul 21 13:36:56.075428 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:56.075383    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c" event={"ID":"ff3e716a-13dc-464e-8be7-c7503dc1bce6","Type":"ContainerStarted","Data":"53e3339b2cf03f49b14ce27a778449723f72449578886dd3b6a9d8b2873e6f35"}
Jul 21 13:36:56.099598 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:56.099543    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c" podStartSLOduration=1.099526317 podStartE2EDuration="1.099526317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:36:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:36:56.0983158 +0000 UTC m=+2748.623818947" watchObservedRunningTime="2026-07-21 13:36:56.099526317 +0000 UTC m=+2748.625029472"
Jul 21 13:36:57.080703 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:57.080659    2563 generic.go:356] "Generic (PLEG): container finished" podID="ff3e716a-13dc-464e-8be7-c7503dc1bce6" containerID="c28e5da0c6302b630c41832adc0d065554b585fbde1e99e90e898d5649d896d3" exitCode=0
Jul 21 13:36:57.081172 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:57.080736    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c" event={"ID":"ff3e716a-13dc-464e-8be7-c7503dc1bce6","Type":"ContainerDied","Data":"c28e5da0c6302b630c41832adc0d065554b585fbde1e99e90e898d5649d896d3"}
Jul 21 13:36:58.222387 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:58.222360    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c"
Jul 21 13:36:58.330358 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:58.330324    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3e716a-13dc-464e-8be7-c7503dc1bce6-maas-api-service-ca\") pod \"ff3e716a-13dc-464e-8be7-c7503dc1bce6\" (UID: \"ff3e716a-13dc-464e-8be7-c7503dc1bce6\") "
Jul 21 13:36:58.330697 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:58.330673    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff3e716a-13dc-464e-8be7-c7503dc1bce6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff3e716a-13dc-464e-8be7-c7503dc1bce6" (UID: "ff3e716a-13dc-464e-8be7-c7503dc1bce6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:36:58.431569 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:58.431527    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3e716a-13dc-464e-8be7-c7503dc1bce6-maas-api-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:36:59.090553 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:59.090518    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c"
Jul 21 13:36:59.090553 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:59.090538    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c" event={"ID":"ff3e716a-13dc-464e-8be7-c7503dc1bce6","Type":"ContainerDied","Data":"53e3339b2cf03f49b14ce27a778449723f72449578886dd3b6a9d8b2873e6f35"}
Jul 21 13:36:59.090781 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:36:59.090590    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53e3339b2cf03f49b14ce27a778449723f72449578886dd3b6a9d8b2873e6f35"
Jul 21 13:37:05.581479 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:05.581440    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"]
Jul 21 13:37:05.582198 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:05.582161    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck" podUID="d60f55ed-bc62-47dc-9312-53c0da604364" containerName="maas-api" containerID="cri-o://75774dccd640bfc0fae702181011a6e7e62d15458aeb261e01e33e3e645b96df" gracePeriod=30
Jul 21 13:37:05.606084 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:05.606052    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:37:06.044833 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.044802    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:37:06.102889 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.102847    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bc52n\" (UniqueName: \"kubernetes.io/projected/d60f55ed-bc62-47dc-9312-53c0da604364-kube-api-access-bc52n\") pod \"d60f55ed-bc62-47dc-9312-53c0da604364\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") "
Jul 21 13:37:06.103092 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.102901    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d60f55ed-bc62-47dc-9312-53c0da604364-maas-api-tls\") pod \"d60f55ed-bc62-47dc-9312-53c0da604364\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") "
Jul 21 13:37:06.103092 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.102931    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d60f55ed-bc62-47dc-9312-53c0da604364-openshift-service-ca\") pod \"d60f55ed-bc62-47dc-9312-53c0da604364\" (UID: \"d60f55ed-bc62-47dc-9312-53c0da604364\") "
Jul 21 13:37:06.103408 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.103381    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d60f55ed-bc62-47dc-9312-53c0da604364-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d60f55ed-bc62-47dc-9312-53c0da604364" (UID: "d60f55ed-bc62-47dc-9312-53c0da604364"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:37:06.105266 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.105242    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d60f55ed-bc62-47dc-9312-53c0da604364-kube-api-access-bc52n" (OuterVolumeSpecName: "kube-api-access-bc52n") pod "d60f55ed-bc62-47dc-9312-53c0da604364" (UID: "d60f55ed-bc62-47dc-9312-53c0da604364"). InnerVolumeSpecName "kube-api-access-bc52n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:37:06.105363 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.105242    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d60f55ed-bc62-47dc-9312-53c0da604364-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d60f55ed-bc62-47dc-9312-53c0da604364" (UID: "d60f55ed-bc62-47dc-9312-53c0da604364"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:37:06.123001 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.122968    2563 generic.go:356] "Generic (PLEG): container finished" podID="d60f55ed-bc62-47dc-9312-53c0da604364" containerID="75774dccd640bfc0fae702181011a6e7e62d15458aeb261e01e33e3e645b96df" exitCode=0
Jul 21 13:37:06.123188 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.123012    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"
Jul 21 13:37:06.123188 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.123048    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck" event={"ID":"d60f55ed-bc62-47dc-9312-53c0da604364","Type":"ContainerDied","Data":"75774dccd640bfc0fae702181011a6e7e62d15458aeb261e01e33e3e645b96df"}
Jul 21 13:37:06.123188 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.123109    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck" event={"ID":"d60f55ed-bc62-47dc-9312-53c0da604364","Type":"ContainerDied","Data":"2657e69fd7427f2d83a75c3d43b40ecac50333f12188b0721ea7490d68dc6965"}
Jul 21 13:37:06.123188 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.123139    2563 scope.go:117] "RemoveContainer" containerID="75774dccd640bfc0fae702181011a6e7e62d15458aeb261e01e33e3e645b96df"
Jul 21 13:37:06.131876 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.131859    2563 scope.go:117] "RemoveContainer" containerID="75774dccd640bfc0fae702181011a6e7e62d15458aeb261e01e33e3e645b96df"
Jul 21 13:37:06.132165 ip-10-0-128-74 kubenswrapper[2563]: E0721 13:37:06.132146    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75774dccd640bfc0fae702181011a6e7e62d15458aeb261e01e33e3e645b96df\": container with ID starting with 75774dccd640bfc0fae702181011a6e7e62d15458aeb261e01e33e3e645b96df not found: ID does not exist" containerID="75774dccd640bfc0fae702181011a6e7e62d15458aeb261e01e33e3e645b96df"
Jul 21 13:37:06.132206 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.132176    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75774dccd640bfc0fae702181011a6e7e62d15458aeb261e01e33e3e645b96df"} err="failed to get container status \"75774dccd640bfc0fae702181011a6e7e62d15458aeb261e01e33e3e645b96df\": rpc error: code = NotFound desc = could not find container \"75774dccd640bfc0fae702181011a6e7e62d15458aeb261e01e33e3e645b96df\": container with ID starting with 75774dccd640bfc0fae702181011a6e7e62d15458aeb261e01e33e3e645b96df not found: ID does not exist"
Jul 21 13:37:06.147040 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.147006    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"]
Jul 21 13:37:06.153598 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.153564    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2df168-844c5c9cbf-ngwck"]
Jul 21 13:37:06.204228 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.204192    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bc52n\" (UniqueName: \"kubernetes.io/projected/d60f55ed-bc62-47dc-9312-53c0da604364-kube-api-access-bc52n\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:37:06.204228 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.204224    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d60f55ed-bc62-47dc-9312-53c0da604364-maas-api-tls\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:37:06.204228 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:06.204237    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d60f55ed-bc62-47dc-9312-53c0da604364-openshift-service-ca\") on node \"ip-10-0-128-74.ec2.internal\" DevicePath \"\""
Jul 21 13:37:07.996639 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:07.996602    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d60f55ed-bc62-47dc-9312-53c0da604364" path="/var/lib/kubelet/pods/d60f55ed-bc62-47dc-9312-53c0da604364/volumes"
Jul 21 13:37:10.253025 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:10.252984    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c"]
Jul 21 13:37:10.254111 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:10.254086    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:37:10.257199 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:10.257171    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2df168-r7l4c"]
Jul 21 13:37:10.635748 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:10.635707    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:37:11.996070 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:11.996021    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff3e716a-13dc-464e-8be7-c7503dc1bce6" path="/var/lib/kubelet/pods/ff3e716a-13dc-464e-8be7-c7503dc1bce6/volumes"
Jul 21 13:37:16.796237 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:16.796196    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:37:16.863045 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:16.863010    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:37:17.098152 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:17.098028    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:37:17.281586 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:17.281551    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:37:23.504947 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:23.504917    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f884b5-jjwjp_825324e5-8fed-402e-9006-0ff3ddd0ec07/copy-manifests/0.log"
Jul 21 13:37:23.518239 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:23.518201    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f884b5-jjwjp_825324e5-8fed-402e-9006-0ff3ddd0ec07/manager/0.log"
Jul 21 13:37:24.423006 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:24.422973    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744010-njbtm_36e56196-c194-4b9f-bbea-57badedf0e47/cleanup/0.log"
Jul 21 13:37:26.902339 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:26.902273    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678jb2hm_e73d90e5-0737-4c57-9ecd-6fb323e6c11f/istio-proxy/0.log"
Jul 21 13:37:27.014422 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:27.014390    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-s5xhb_15780951-a3f8-45b5-ae67-1a8e47d0462a/discovery/0.log"
Jul 21 13:37:27.347771 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:27.347679    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xxfdb_ff529716-9b98-4184-b2ef-9ae9e1b3e428/istio-proxy/1.log"
Jul 21 13:37:27.579745 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:27.579713    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-8xf42_55144294-dceb-4f6b-95fc-921511fc0c19/payload-processing/0.log"
Jul 21 13:37:27.687586 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:27.687559    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-785f46d866-4qbft_746485ca-3bb7-46e3-82b8-e9bdebf3ad96/router/0.log"
Jul 21 13:37:28.126280 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:28.126245    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-rkjll_d84addf1-9568-4631-8ca9-78fad1357703/main/0.log"
Jul 21 13:37:28.453682 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:28.453597    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-vd68l_150aa186-45f3-42a3-ab16-49a29fdaf457/main/0.log"
Jul 21 13:37:34.407893 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:34.407858    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:37:34.592110 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:34.592076    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log" limit=1024
Jul 21 13:37:37.564728 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:37.564692    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n7pwd_49c51a2b-86c0-4f43-af87-0f9dc3a56986/global-pull-secret-syncer/0.log"
Jul 21 13:37:37.620309 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:37.620256    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5w7wb_50c5c92e-e9f6-4856-bc1d-4a6ef81c7f47/konnectivity-agent/0.log"
Jul 21 13:37:37.752689 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:37.752658    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-74.ec2.internal_b98dd9609e64d70799acdc2e03bf5112/haproxy/0.log"
Jul 21 13:37:45.356941 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:45.356898    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-b5v7d_41f99d1e-551f-457e-8a74-87f9702496cf/cluster-monitoring-operator/0.log"
Jul 21 13:37:45.711369 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:45.711273    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hbqlt_66096f3e-3cfa-48e0-ad97-0104d69436ea/node-exporter/0.log"
Jul 21 13:37:45.738177 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:45.738148    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hbqlt_66096f3e-3cfa-48e0-ad97-0104d69436ea/kube-rbac-proxy/0.log"
Jul 21 13:37:45.763470 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:45.763444    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hbqlt_66096f3e-3cfa-48e0-ad97-0104d69436ea/init-textfile/0.log"
Jul 21 13:37:47.739960 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.739926    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"]
Jul 21 13:37:47.740375 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.740228    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff3e716a-13dc-464e-8be7-c7503dc1bce6" containerName="revoke-keys"
Jul 21 13:37:47.740375 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.740240    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff3e716a-13dc-464e-8be7-c7503dc1bce6" containerName="revoke-keys"
Jul 21 13:37:47.740375 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.740258    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d60f55ed-bc62-47dc-9312-53c0da604364" containerName="maas-api"
Jul 21 13:37:47.740375 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.740264    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d60f55ed-bc62-47dc-9312-53c0da604364" containerName="maas-api"
Jul 21 13:37:47.740375 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.740330    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d60f55ed-bc62-47dc-9312-53c0da604364" containerName="maas-api"
Jul 21 13:37:47.740375 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.740341    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff3e716a-13dc-464e-8be7-c7503dc1bce6" containerName="revoke-keys"
Jul 21 13:37:47.743854 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.743813    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.748023 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.747999    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4m5mj\"/\"kube-root-ca.crt\""
Jul 21 13:37:47.749133 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.749112    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4m5mj\"/\"default-dockercfg-565cz\""
Jul 21 13:37:47.749259 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.749112    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4m5mj\"/\"openshift-service-ca.crt\""
Jul 21 13:37:47.766471 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.766432    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-qhqc5_f2b7850b-0f49-4bb4-8de3-4a8a2396a9ee/networking-console-plugin/0.log"
Jul 21 13:37:47.767117 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.767095    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"]
Jul 21 13:37:47.888166 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.888122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e7e34234-563c-4f26-a5dd-7c366c00e242-proc\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.888166 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.888164    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e7e34234-563c-4f26-a5dd-7c366c00e242-podres\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.888426 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.888191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7e34234-563c-4f26-a5dd-7c366c00e242-lib-modules\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.888426 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.888274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7e34234-563c-4f26-a5dd-7c366c00e242-sys\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.888426 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.888362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksh8j\" (UniqueName: \"kubernetes.io/projected/e7e34234-563c-4f26-a5dd-7c366c00e242-kube-api-access-ksh8j\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.989224 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.989185    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e7e34234-563c-4f26-a5dd-7c366c00e242-proc\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.989224 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.989227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e7e34234-563c-4f26-a5dd-7c366c00e242-podres\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.989469 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.989253    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7e34234-563c-4f26-a5dd-7c366c00e242-lib-modules\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.989469 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.989325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7e34234-563c-4f26-a5dd-7c366c00e242-sys\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.989469 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.989341    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e7e34234-563c-4f26-a5dd-7c366c00e242-proc\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.989469 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.989396    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7e34234-563c-4f26-a5dd-7c366c00e242-lib-modules\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.989469 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.989403    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksh8j\" (UniqueName: \"kubernetes.io/projected/e7e34234-563c-4f26-a5dd-7c366c00e242-kube-api-access-ksh8j\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.989469 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.989404    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7e34234-563c-4f26-a5dd-7c366c00e242-sys\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.989469 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.989469    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e7e34234-563c-4f26-a5dd-7c366c00e242-podres\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:47.999992 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:47.999922    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksh8j\" (UniqueName: \"kubernetes.io/projected/e7e34234-563c-4f26-a5dd-7c366c00e242-kube-api-access-ksh8j\") pod \"perf-node-gather-daemonset-vvxxx\" (UID: \"e7e34234-563c-4f26-a5dd-7c366c00e242\") " pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:48.055870 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:48.055832    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:48.202254 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:48.202226    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"]
Jul 21 13:37:48.204269 ip-10-0-128-74 kubenswrapper[2563]: W0721 13:37:48.204233    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode7e34234_563c_4f26_a5dd_7c366c00e242.slice/crio-1cd30727299c2cf6466669735dffc03e3b64580f2af3e53e2acae8843545457b WatchSource:0}: Error finding container 1cd30727299c2cf6466669735dffc03e3b64580f2af3e53e2acae8843545457b: Status 404 returned error can't find the container with id 1cd30727299c2cf6466669735dffc03e3b64580f2af3e53e2acae8843545457b
Jul 21 13:37:48.306721 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:48.306687    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx" event={"ID":"e7e34234-563c-4f26-a5dd-7c366c00e242","Type":"ContainerStarted","Data":"918e63641a36ceb8edb6df554ed263ebc319f8dd647bc600d8d1a65b0374c5e4"}
Jul 21 13:37:48.306901 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:48.306739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx" event={"ID":"e7e34234-563c-4f26-a5dd-7c366c00e242","Type":"ContainerStarted","Data":"1cd30727299c2cf6466669735dffc03e3b64580f2af3e53e2acae8843545457b"}
Jul 21 13:37:48.306901 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:48.306815    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:48.331630 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:48.331577    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx" podStartSLOduration=1.331560682 podStartE2EDuration="1.331560682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:37:48.330896932 +0000 UTC m=+2800.856400088" watchObservedRunningTime="2026-07-21 13:37:48.331560682 +0000 UTC m=+2800.857063837"
Jul 21 13:37:49.067772 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:49.067742    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-d5ptb_9d55962c-a472-465a-8285-9c8bf0856ae4/download-server/0.log"
Jul 21 13:37:51.149738 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:51.149703    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rtn7b_698e2686-c7ef-4e97-bed4-7cc24e1bd4fe/dns/0.log"
Jul 21 13:37:51.183191 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:51.183147    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rtn7b_698e2686-c7ef-4e97-bed4-7cc24e1bd4fe/kube-rbac-proxy/0.log"
Jul 21 13:37:51.294580 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:51.294546    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w92wm_fcebc074-dea4-4fe1-9208-f01443f8720b/dns-node-resolver/0.log"
Jul 21 13:37:51.936864 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:51.936828    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6bbc7f78d5-cfgw7_c5394394-40bb-4611-b59b-40721647c330/registry/0.log"
Jul 21 13:37:52.063256 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:52.063225    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nln7n_5bc0b65d-775f-43f5-b762-26ae86e132d1/node-ca/0.log"
Jul 21 13:37:53.235550 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:53.235518    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678jb2hm_e73d90e5-0737-4c57-9ecd-6fb323e6c11f/istio-proxy/0.log"
Jul 21 13:37:53.389565 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:53.389534    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-s5xhb_15780951-a3f8-45b5-ae67-1a8e47d0462a/discovery/0.log"
Jul 21 13:37:53.501468 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:53.501380    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xxfdb_ff529716-9b98-4184-b2ef-9ae9e1b3e428/istio-proxy/0.log"
Jul 21 13:37:53.527141 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:53.527109    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xxfdb_ff529716-9b98-4184-b2ef-9ae9e1b3e428/istio-proxy/1.log"
Jul 21 13:37:53.614779 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:53.614746    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-8xf42_55144294-dceb-4f6b-95fc-921511fc0c19/payload-processing/0.log"
Jul 21 13:37:53.643864 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:53.643837    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-785f46d866-4qbft_746485ca-3bb7-46e3-82b8-e9bdebf3ad96/router/0.log"
Jul 21 13:37:54.322091 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:54.322062    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4m5mj/perf-node-gather-daemonset-vvxxx"
Jul 21 13:37:54.454505 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:54.454470    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b6h6z_2726d88e-bdde-4182-a2a5-2a1224965db8/serve-healthcheck-canary/0.log"
Jul 21 13:37:55.183244 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:55.183217    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-pnxvv_d12b1d08-a810-4d9a-bcc8-7063e100a963/insights-operator/0.log"
Jul 21 13:37:55.473239 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:55.473157    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kv8z4_66a4327a-f8c0-4865-834d-f964c9b00a75/kube-rbac-proxy/0.log"
Jul 21 13:37:55.516786 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:55.516757    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kv8z4_66a4327a-f8c0-4865-834d-f964c9b00a75/exporter/0.log"
Jul 21 13:37:55.552431 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:55.552399    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kv8z4_66a4327a-f8c0-4865-834d-f964c9b00a75/extractor/0.log"
Jul 21 13:37:58.057851 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:58.057761    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678jb2hm_e73d90e5-0737-4c57-9ecd-6fb323e6c11f/istio-proxy/0.log"
Jul 21 13:37:58.180075 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:58.180044    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-s5xhb_15780951-a3f8-45b5-ae67-1a8e47d0462a/discovery/0.log"
Jul 21 13:37:58.248914 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:58.248876    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xxfdb_ff529716-9b98-4184-b2ef-9ae9e1b3e428/istio-proxy/0.log"
Jul 21 13:37:58.275218 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:58.275189    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xxfdb_ff529716-9b98-4184-b2ef-9ae9e1b3e428/istio-proxy/1.log"
Jul 21 13:37:58.335107 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:58.335013    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-8xf42_55144294-dceb-4f6b-95fc-921511fc0c19/payload-processing/0.log"
Jul 21 13:37:58.360887 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:58.360857    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-785f46d866-4qbft_746485ca-3bb7-46e3-82b8-e9bdebf3ad96/router/0.log"
Jul 21 13:37:59.047014 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:59.046975    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f884b5-jjwjp_825324e5-8fed-402e-9006-0ff3ddd0ec07/manager/0.log"
Jul 21 13:37:59.079722 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:59.079697    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f884b5-jjwjp_825324e5-8fed-402e-9006-0ff3ddd0ec07/copy-manifests/0.log"
Jul 21 13:37:59.652187 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:59.652152    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678jb2hm_e73d90e5-0737-4c57-9ecd-6fb323e6c11f/istio-proxy/0.log"
Jul 21 13:37:59.791079 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:59.790960    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-s5xhb_15780951-a3f8-45b5-ae67-1a8e47d0462a/discovery/0.log"
Jul 21 13:37:59.887735 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:59.887703    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xxfdb_ff529716-9b98-4184-b2ef-9ae9e1b3e428/istio-proxy/0.log"
Jul 21 13:37:59.914604 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:59.914571    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-xxfdb_ff529716-9b98-4184-b2ef-9ae9e1b3e428/istio-proxy/1.log"
Jul 21 13:37:59.969657 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:59.969620    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-8xf42_55144294-dceb-4f6b-95fc-921511fc0c19/payload-processing/0.log"
Jul 21 13:37:59.987702 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:37:59.987670    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-785f46d866-4qbft_746485ca-3bb7-46e3-82b8-e9bdebf3ad96/router/0.log"
Jul 21 13:38:00.548652 ip-10-0-128-74 kubenswrapper[2563]: I0721 13:38:00.548627    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-54f95887b7-ckjhk_913e9809-59dd-42c1-94ce-15bbd88fdb57/manager/0.log"