Jul 28 04:11:44.938618 ip-10-0-133-204 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 04:11:45.336723 ip-10-0-133-204 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 04:11:45.336723 ip-10-0-133-204 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 04:11:45.336723 ip-10-0-133-204 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 04:11:45.336723 ip-10-0-133-204 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 04:11:45.336723 ip-10-0-133-204 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 04:11:45.338353 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.338281    2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 04:11:45.342633 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342620    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 04:11:45.342633 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342633    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 04:11:45.342633 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342636    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342641    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342644    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342647    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342650    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342653    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342656    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342659    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342662    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342665    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342668    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342671    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342674    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342676    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342679    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342682    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342684    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342687    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342690    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342693    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 04:11:45.342731 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342695    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342698    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342701    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342703    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342706    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342708    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342711    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342714    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342718    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342721    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342724    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342726    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342729    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342732    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342735    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342738    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342741    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342744    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342747    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 04:11:45.343202 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342751    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342754    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342757    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342760    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342763    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342771    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342774    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342777    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342780    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342782    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342785    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342788    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342790    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342793    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342795    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342798    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342800    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342803    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342805    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342808    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 04:11:45.343714 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342810    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342813    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342815    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342817    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342820    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342823    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342826    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342829    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342832    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342834    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342838    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342841    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342843    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342846    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342848    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342851    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342853    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342855    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342858    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342860    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 04:11:45.344193 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342863    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342865    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342867    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342870    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342872    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342874    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342877    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342880    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342882    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342886    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342889    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342892    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342895    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342898    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342901    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342904    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342907    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.342909    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343580    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 04:11:45.344711 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343588    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343592    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343595    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343599    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343603    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343606    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343610    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343613    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343616    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343619    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343621    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343624    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343626    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343629    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343631    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343634    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343636    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343639    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343641    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343644    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 04:11:45.345162 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343646    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343649    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343652    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343655    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343657    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343660    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343662    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343665    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343667    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343670    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343673    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343676    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343678    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343681    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343683    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343686    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343688    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343691    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343694    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343697    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 04:11:45.345659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343699    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343702    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343705    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343707    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343710    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343712    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343715    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343719    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343721    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343724    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343727    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343729    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343732    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343736    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343739    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343742    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343745    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343747    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343750    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343753    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 04:11:45.346147 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343756    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343758    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343761    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343763    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343766    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343768    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343771    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343773    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343775    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343778    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343781    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343783    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343786    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343788    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343791    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343793    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343796    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343800    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343802    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343805    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 04:11:45.346659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343808    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343810    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343813    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343815    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343818    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343820    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343823    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343825    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343828    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343830    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343833    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343835    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343838    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343841    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343843    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343846    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343848    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.343850    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344194    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 04:11:45.347165 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344198    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344201    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344204    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344207    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344210    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344212    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344215    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344217    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344220    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344223    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344225    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344228    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344231    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344234    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344238    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344241    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344254    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344257    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344259    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344262    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 04:11:45.347728 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344265    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344267    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344270    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344273    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344276    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344279    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344281    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344284    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344286    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344289    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344293    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344297    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344299    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344302    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344305    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344307    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344311    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344313    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344316    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344319    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 04:11:45.348233 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344321    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344324    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344326    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344329    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344332    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344334    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344337    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344339    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344342    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344344    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344347    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344349    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344352    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344355    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344369    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344373    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344375    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344378    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344380    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344383    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 04:11:45.348778 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344386    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344388    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344391    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344393    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344396    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344400    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344403    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344406    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344409    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344412    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344414    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344417    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344419    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344422    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344424    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344429    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344432    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344437    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344439    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 04:11:45.349259 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344442    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344444    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344447    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344449    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344452    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344455    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344458    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344460    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344463    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344465    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344468    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344470    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344473    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344475    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344478    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344480    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344483    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344485    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.344488    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344554    2574 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 04:11:45.349725 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344560    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344566    2574 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344570    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344575    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344578    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344582    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344590    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344593    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344598    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344601    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344606    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344609    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344612    2574 flags.go:64] FLAG: --cgroup-root=""
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344614    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344617    2574 flags.go:64] FLAG: --client-ca-file=""
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344620    2574 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344623    2574 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344627    2574 flags.go:64] FLAG: --cluster-domain=""
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344629    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344632    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344635    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344639    2574 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344642    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 04:11:45.350238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344646    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344649    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344652    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344655    2574 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344657    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344660    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344664    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344666    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344670    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344673    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344676    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344679    2574 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344682    2574 flags.go:64] FLAG: --enable-server="true"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344685    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344689    2574 flags.go:64] FLAG: --event-burst="100"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344692    2574 flags.go:64] FLAG: --event-qps="50"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344695    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344699    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344701    2574 flags.go:64] FLAG: --eviction-hard=""
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344706    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344709    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344712    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344715    2574 flags.go:64] FLAG: --eviction-soft=""
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344718    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344721    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 04:11:45.350807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344724    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344727    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344730    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344732    2574 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344735    2574 flags.go:64] FLAG: --feature-gates=""
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344739    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344742    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344745    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344748    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344751    2574 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344753    2574 flags.go:64] FLAG: --help="false"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344756    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344759    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344762    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344765    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344768    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344771    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344774    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344776    2574 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344779    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344782    2574 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344785    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344788    2574 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344791    2574 flags.go:64] FLAG: --kube-reserved=""
Jul 28 04:11:45.351433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344795    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344797    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344801    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344804    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344806    2574 flags.go:64] FLAG: --lock-file=""
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344809    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344812    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344815    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344820    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344823    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344825    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344828    2574 flags.go:64] FLAG: --logging-format="text"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344831    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344834    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344836    2574 flags.go:64] FLAG: --manifest-url=""
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344839    2574 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344843    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344846    2574 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344850    2574 flags.go:64] FLAG: --max-pods="110"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344853    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344856    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344858    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344861    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344864    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344867    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 04:11:45.352013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344869    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344876    2574 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344879    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344881    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344884    2574 flags.go:64] FLAG: --pod-cidr=""
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344888    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344893    2574 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344897    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344899    2574 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344902    2574 flags.go:64] FLAG: --port="10250"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344905    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344908    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ed72cacfa33decf0"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344911    2574 flags.go:64] FLAG: --qos-reserved=""
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344914    2574 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344917    2574 flags.go:64] FLAG: --register-node="true"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344920    2574 flags.go:64] FLAG: --register-with-taints=""
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344923    2574 flags.go:64] FLAG: --registry-burst="10"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344926    2574 flags.go:64] FLAG: --registry-qps="5"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344928    2574 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344931    2574 flags.go:64] FLAG: --reserved-memory=""
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344935    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344937    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344940    2574 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344943    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344946    2574 flags.go:64] FLAG: --runonce="false"
Jul 28 04:11:45.352636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344948    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344952    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344954    2574 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344957    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344960    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344963    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344966    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344968    2574 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344971    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344974    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344976    2574 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344979    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344982    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344985    2574 flags.go:64] FLAG: --system-cgroups=""
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344989    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344994    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.344997    2574 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.345000    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.345003    2574 flags.go:64] FLAG: --tls-min-version=""
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.345006    2574 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.345009    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.345012    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.345014    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.345017    2574 flags.go:64] FLAG: --v="2"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.345021    2574 flags.go:64] FLAG: --version="false"
Jul 28 04:11:45.353240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.345025    2574 flags.go:64] FLAG: --vmodule=""
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.345028    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.345031    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345134    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345138    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345142    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345145    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345148    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345150    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345153    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345156    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345158    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345161    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345164    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345167    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345169    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345172    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345175    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345177    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345180    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 04:11:45.353889 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345182    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345187    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345189    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345192    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345196    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345199    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345202    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345205    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345208    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345211    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345213    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345216    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345218    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345221    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345223    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345226    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345228    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345231    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345233    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345236    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 04:11:45.354388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345238    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345240    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345243    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345245    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345248    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345250    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345252    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345255    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345257    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345260    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345262    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345264    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345267    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345271    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345274    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345278    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345281    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345283    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345286    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345289    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 04:11:45.354882 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345291    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345293    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345296    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345300    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345303    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345306    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345309    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345312    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345315    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345317    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345320    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345322    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345325    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345327    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345330    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345332    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345335    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345337    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345340    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 04:11:45.355380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345342    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345345    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345347    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345349    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345352    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345355    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345370    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345373    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345378    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345381    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345383    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345386    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345389    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345392    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345394    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345398    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345402    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345405    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345407    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 04:11:45.355859 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345410    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345413    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345416    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.345418    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.345424    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.351638    2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.351650    2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.351667    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.351674    2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351724    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351728    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351732    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351735    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351737    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351740    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 04:11:45.356334 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351743    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351746    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351749    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351753    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351758    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351762    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351765    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351769    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351772    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351775    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351788    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351791    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351794    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351796    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351799    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351802    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351804    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351807    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351810    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 04:11:45.356712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351812    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351815    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351817    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351820    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351822    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351825    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351827    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351830    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351832    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351835    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351837    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351840    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351843    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351845    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351848    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351850    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351853    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351856    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351858    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351861    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 04:11:45.357167 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351864    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351867    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351870    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351872    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351875    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351877    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351880    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351882    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351885    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351887    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351890    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351892    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351895    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351898    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351900    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351903    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351905    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351908    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351910    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351913    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351915    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 04:11:45.357673 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351918    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351920    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351923    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351926    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351928    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351930    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351933    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351936    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351938    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351941    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351944    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351946    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351949    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351952    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351956    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351959    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351963    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351965    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351968    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 04:11:45.358535 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351970    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351973    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351975    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351978    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351981    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351983    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351986    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351989    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351991    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351994    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351996    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.351999    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352001    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352004    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.352009    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 04:11:45.359274 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352113    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352117    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352120    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352122    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352125    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352127    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352130    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352133    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352135    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352138    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352140    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352143    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352146    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352148    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352151    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352153    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352156    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352158    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352161    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 04:11:45.359671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352163    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352166    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352169    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352171    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352174    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352176    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352179    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352181    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352184    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352186    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352189    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352191    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352194    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352196    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352199    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352201    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352204    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352206    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352209    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352211    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 04:11:45.360134 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352214    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352216    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352219    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352222    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352224    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352227    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352230    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352232    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352234    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352237    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352239    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352242    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352252    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352257    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352260    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352262    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352265    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352268    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352271    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352274    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 04:11:45.360652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352277    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352279    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352282    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352285    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352287    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352289    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352292    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352294    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352297    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352299    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352302    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352304    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352307    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352309    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352312    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352314    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352316    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352319    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352322    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352325    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 04:11:45.361129 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352327    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352329    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352332    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352334    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352337    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352341    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352343    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352346    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352348    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352352    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352356    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352371    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352375    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352378    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352381    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352384    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352386    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352389    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352391    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 04:11:45.361786 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:45.352394    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 04:11:45.362302 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.352399    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 04:11:45.362302 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.352488    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 04:11:45.362302 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.354324    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 04:11:45.362302 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.355571    2574 server.go:1019] "Starting client certificate rotation"
Jul 28 04:11:45.362302 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.355667    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 04:11:45.362302 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.355706    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 04:11:45.362302 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.358911    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 04:11:45.362302 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.361251    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 04:11:45.371071 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.371053    2574 log.go:25] "Validated CRI v1 runtime API"
Jul 28 04:11:45.374218 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.374201    2574 log.go:25] "Validated CRI v1 image API"
Jul 28 04:11:45.374997 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.374982    2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 04:11:45.380453 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.380426    2574 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b2379f66-4fbd-4cdc-bbef-093251dac7b7:/dev/nvme0n1p3 e1fcf53b-e7a6-4331-b560-5b556a1f48ba:/dev/nvme0n1p4]
Jul 28 04:11:45.380540 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.380444    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 04:11:45.384018 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.383996    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 04:11:45.384616 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.384516    2574 manager.go:216] Machine: {Timestamp:2026-07-28 04:11:45.383995515 +0000 UTC m=+0.325835226 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102559 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25efd5fcda55d450a98f303081a2db SystemUUID:ec25efd5-fcda-55d4-50a9-8f303081a2db BootID:89366b47-2195-4bec-9ae3-a259e38fdc79 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:89:9d:6b:65:03 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:89:9d:6b:65:03 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:42:2e:71:96:2a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 04:11:45.384616 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.384608    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 04:11:45.384753 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.384696    2574 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 04:11:45.387799 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.387771    2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 04:11:45.387947 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.387801    2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-204.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 04:11:45.388512 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.388500    2574 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 04:11:45.388566 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.388517    2574 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 04:11:45.388566 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.388527    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 04:11:45.388566 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.388541    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 04:11:45.389716 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.389704    2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 04:11:45.392634 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.392621    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 04:11:45.392787 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.392775    2574 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 04:11:45.392843 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.392811    2574 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 04:11:45.392843 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.392833    2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 04:11:45.392935 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.392847    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 04:11:45.392935 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.392862    2574 kubelet.go:398] "Adding apiserver pod source"
Jul 28 04:11:45.392935 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.392874    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 04:11:45.393236 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.393223    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 04:11:45.393303 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.393246    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 04:11:45.396017 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.396000    2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 04:11:45.397679 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.397665    2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 04:11:45.397728 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.397694    2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 04:11:45.399987 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.399961    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 04:11:45.399987 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.399985    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 04:11:45.400133 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.399994    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 04:11:45.400133 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.400003    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 04:11:45.400133 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.400016    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 04:11:45.400133 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.400032    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 04:11:45.400133 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.400044    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 04:11:45.400133 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.400052    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 04:11:45.400133 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.400063    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 04:11:45.400133 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.400072    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 04:11:45.400133 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.400103    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 04:11:45.400133 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.400114    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 04:11:45.400963 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.400948    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 04:11:45.401059 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.401050    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 04:11:45.403216 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.403199    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-204.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 04:11:45.403289 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.403211    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 04:11:45.403289 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.403220    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 04:11:45.404527 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.404511    2574 server.go:1268] "Started kubelet"
Jul 28 04:11:45.404598 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.404565    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 04:11:45.404647 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.404601    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 04:11:45.404695 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.404665    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 04:11:45.405246 ip-10-0-133-204 systemd[1]: Started Kubernetes Kubelet.
Jul 28 04:11:45.405775 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.405750    2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 04:11:45.407007 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.406989    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9vf22"
Jul 28 04:11:45.407240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.407227    2574 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 04:11:45.409199 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.408274    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-204.ec2.internal.18c658c4fe6eb925  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-204.ec2.internal,UID:ip-10-0-133-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-204.ec2.internal,},FirstTimestamp:2026-07-28 04:11:45.404483877 +0000 UTC m=+0.346323588,LastTimestamp:2026-07-28 04:11:45.404483877 +0000 UTC m=+0.346323588,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-204.ec2.internal,}"
Jul 28 04:11:45.409934 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.409916    2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 04:11:45.411878 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.411864    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 04:11:45.411936 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.411890    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 04:11:45.411936 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.411907    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9vf22"
Jul 28 04:11:45.412134 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.412087    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-204.ec2.internal\" not found"
Jul 28 04:11:45.412469 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.412448    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 04:11:45.412469 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.412469    2574 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 04:11:45.412618 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.412483    2574 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 04:11:45.412618 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.412546    2574 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 04:11:45.412618 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.412554    2574 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 04:11:45.412769 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.412667    2574 factory.go:55] Registering systemd factory
Jul 28 04:11:45.412769 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.412682    2574 factory.go:223] Registration of the systemd container factory successfully
Jul 28 04:11:45.412872 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.412860    2574 factory.go:153] Registering CRI-O factory
Jul 28 04:11:45.412966 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.412874    2574 factory.go:223] Registration of the crio container factory successfully
Jul 28 04:11:45.412966 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.412934    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 04:11:45.412966 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.412958    2574 factory.go:103] Registering Raw factory
Jul 28 04:11:45.413083 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.412972    2574 manager.go:1196] Started watching for new ooms in manager
Jul 28 04:11:45.413571 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.413555    2574 manager.go:349] Starting recovery of all containers
Jul 28 04:11:45.423099 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.423077    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 04:11:45.424330 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.424313    2574 manager.go:354] Recovery completed
Jul 28 04:11:45.426001 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.425880    2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-204.ec2.internal\" not found" node="ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.429605 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.429591    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 04:11:45.431418 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.431404    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 04:11:45.431488 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.431431    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 04:11:45.431488 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.431444    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientPID"
Jul 28 04:11:45.431924 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.431910    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 04:11:45.431924 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.431923    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 04:11:45.432025 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.431938    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 04:11:45.432025 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.431944    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 04:11:45.432864 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.432852    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 04:11:45.432910 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.432873    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 04:11:45.432910 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.432882    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientPID"
Jul 28 04:11:45.433461 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.433451    2574 policy_none.go:49] "None policy: Start"
Jul 28 04:11:45.433498 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.433465    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 04:11:45.433498 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.433475    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 04:11:45.434585 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.434575    2574 policy_none.go:47] "Start"
Jul 28 04:11:45.478890 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.478874    2574 manager.go:337] "Starting Device Plugin manager"
Jul 28 04:11:45.478968 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.478901    2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 04:11:45.478968 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.478910    2574 server.go:87] "Starting device plugin registration server"
Jul 28 04:11:45.479119 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.479111    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 04:11:45.479196 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.479121    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 04:11:45.479245 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.479213    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 04:11:45.479299 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.479275    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 04:11:45.479299 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.479285    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 04:11:45.479813 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.479779    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 04:11:45.479887 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.479828    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-204.ec2.internal\" not found"
Jul 28 04:11:45.579517 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.579505    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 04:11:45.581016 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.581003    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 04:11:45.581089 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.581025    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 04:11:45.581089 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.581035    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientPID"
Jul 28 04:11:45.581089 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.581059    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.588326 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.588293    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.588326 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.588312    2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-204.ec2.internal\": node \"ip-10-0-133-204.ec2.internal\" not found"
Jul 28 04:11:45.601621 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.601601    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-204.ec2.internal\" not found"
Jul 28 04:11:45.612776 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.612746    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 04:11:45.613860 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.613846    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 04:11:45.613915 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.613880    2574 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 04:11:45.613915 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.613902    2574 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 04:11:45.613972 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.613931    2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 04:11:45.616270 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.616251    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 04:11:45.702669 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.702646    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-204.ec2.internal\" not found"
Jul 28 04:11:45.714898 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.714881    2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-204.ec2.internal"]
Jul 28 04:11:45.714960 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.714933    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 04:11:45.718033 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.718020    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 04:11:45.718088 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.718044    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 04:11:45.718088 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.718053    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientPID"
Jul 28 04:11:45.719235 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.719223    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 04:11:45.719412 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.719399    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.719456 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.719425    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 04:11:45.720038 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.720025    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 04:11:45.720120 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.720043    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 04:11:45.720120 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.720052    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientPID"
Jul 28 04:11:45.720120 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.720069    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 04:11:45.720120 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.720084    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 04:11:45.720120 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.720095    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientPID"
Jul 28 04:11:45.721245 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.721229    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.721315 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.721252    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 04:11:45.722868 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.722847    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 04:11:45.722938 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.722872    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 04:11:45.722938 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.722881    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeHasSufficientPID"
Jul 28 04:11:45.738875 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.738854    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-204.ec2.internal\" not found" node="ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.742773 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.742756    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-204.ec2.internal\" not found" node="ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.802962 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.802944    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-204.ec2.internal\" not found"
Jul 28 04:11:45.815237 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.815220    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6db9e0d726c4428a7173af2b853a8bd3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-204.ec2.internal\" (UID: \"6db9e0d726c4428a7173af2b853a8bd3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.815300 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.815240    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5e83b5d2069ff1f9446be40c52eae0c3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal\" (UID: \"5e83b5d2069ff1f9446be40c52eae0c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.815300 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.815259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e83b5d2069ff1f9446be40c52eae0c3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal\" (UID: \"5e83b5d2069ff1f9446be40c52eae0c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.903534 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:45.903472    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-204.ec2.internal\" not found"
Jul 28 04:11:45.915708 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.915683    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5e83b5d2069ff1f9446be40c52eae0c3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal\" (UID: \"5e83b5d2069ff1f9446be40c52eae0c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.915769 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.915714    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e83b5d2069ff1f9446be40c52eae0c3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal\" (UID: \"5e83b5d2069ff1f9446be40c52eae0c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.915769 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.915729    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6db9e0d726c4428a7173af2b853a8bd3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-204.ec2.internal\" (UID: \"6db9e0d726c4428a7173af2b853a8bd3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.915769 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.915751    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6db9e0d726c4428a7173af2b853a8bd3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-204.ec2.internal\" (UID: \"6db9e0d726c4428a7173af2b853a8bd3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.915880 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.915776    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e83b5d2069ff1f9446be40c52eae0c3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal\" (UID: \"5e83b5d2069ff1f9446be40c52eae0c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:45.915880 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:45.915778    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5e83b5d2069ff1f9446be40c52eae0c3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal\" (UID: \"5e83b5d2069ff1f9446be40c52eae0c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:46.003798 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:46.003774    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-204.ec2.internal\" not found"
Jul 28 04:11:46.041092 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.041079    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:46.046782 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.046767    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:46.104235 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:46.104212    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-204.ec2.internal\" not found"
Jul 28 04:11:46.204831 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:46.204779    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-204.ec2.internal\" not found"
Jul 28 04:11:46.305190 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:46.305155    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-204.ec2.internal\" not found"
Jul 28 04:11:46.354672 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.354646    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 04:11:46.355077 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.354779    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 04:11:46.355077 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.354804    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 04:11:46.394676 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.394651    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 04:11:46.405352 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:46.405326    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-204.ec2.internal\" not found"
Jul 28 04:11:46.418410 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.418381    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 04:06:45 +0000 UTC" deadline="2028-04-03 09:13:08.157454418 +0000 UTC"
Jul 28 04:11:46.418410 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.418405    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14765h1m21.73905262s"
Jul 28 04:11:46.495629 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.495608    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 04:11:46.512281 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.512261    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:46.524769 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.524752    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 04:11:46.525792 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.525779    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal"
Jul 28 04:11:46.537806 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.537790    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 04:11:46.561902 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:46.561872    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6db9e0d726c4428a7173af2b853a8bd3.slice/crio-be5dd75a2d2bd186de9616d60b6c814e518c42a1253c9ee13f621193a18349e7 WatchSource:0}: Error finding container be5dd75a2d2bd186de9616d60b6c814e518c42a1253c9ee13f621193a18349e7: Status 404 returned error can't find the container with id be5dd75a2d2bd186de9616d60b6c814e518c42a1253c9ee13f621193a18349e7
Jul 28 04:11:46.562070 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:46.562055    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e83b5d2069ff1f9446be40c52eae0c3.slice/crio-248d11c331ee3198922ef877d32c8c3ac650aa2132938b2328382790af622efe WatchSource:0}: Error finding container 248d11c331ee3198922ef877d32c8c3ac650aa2132938b2328382790af622efe: Status 404 returned error can't find the container with id 248d11c331ee3198922ef877d32c8c3ac650aa2132938b2328382790af622efe
Jul 28 04:11:46.565122 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.565108    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 04:11:46.617472 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.617429    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal" event={"ID":"5e83b5d2069ff1f9446be40c52eae0c3","Type":"ContainerStarted","Data":"248d11c331ee3198922ef877d32c8c3ac650aa2132938b2328382790af622efe"}
Jul 28 04:11:46.618354 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:46.618331    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-204.ec2.internal" event={"ID":"6db9e0d726c4428a7173af2b853a8bd3","Type":"ContainerStarted","Data":"be5dd75a2d2bd186de9616d60b6c814e518c42a1253c9ee13f621193a18349e7"}
Jul 28 04:11:47.393650 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.393618    2574 apiserver.go:52] "Watching apiserver"
Jul 28 04:11:47.399777 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.399754    2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 04:11:47.400199 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.400174    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-xgf87","openshift-multus/multus-additional-cni-plugins-mrsls","openshift-multus/multus-zl45v","openshift-multus/network-metrics-daemon-npqcf","kube-system/kube-apiserver-proxy-ip-10-0-133-204.ec2.internal","openshift-image-registry/node-ca-27l4c","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal","openshift-network-diagnostics/network-check-target-5rg2c","openshift-network-operator/iptables-alerter-w4rlp","openshift-ovn-kubernetes/ovnkube-node-tb5v6","kube-system/global-pull-secret-syncer-wgdm4","kube-system/konnectivity-agent-b47jl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk","openshift-cluster-node-tuning-operator/tuned-zq2l4"]
Jul 28 04:11:47.402052 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.402033    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.403347 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.403327    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:47.403472 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:47.403428    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:11:47.404888 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.404866    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 04:11:47.405134 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.405097    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 04:11:47.405134 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.405112    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 04:11:47.405259 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.405228    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 04:11:47.405311 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.405256    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8dr2z\""
Jul 28 04:11:47.405311 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.405103    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 04:11:47.405542 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.405521    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.405972 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.405957    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:47.406057 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:47.406017    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:11:47.406201 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.406181    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 04:11:47.407796 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.407777    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 04:11:47.408083 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.408069    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 04:11:47.408254 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.408240    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 04:11:47.408462 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.408447    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 04:11:47.408635 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.408618    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xmg7q\""
Jul 28 04:11:47.408805 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.408769    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.409234 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.409202    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-27l4c"
Jul 28 04:11:47.411431 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.410378    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 04:11:47.411431 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.410731    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jftcw\""
Jul 28 04:11:47.411431 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.411027    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 04:11:47.411431 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.411403    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 04:11:47.411798 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.411585    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 04:11:47.411798 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.411611    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w4rlp"
Jul 28 04:11:47.411860 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.411815    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bzjvz\""
Jul 28 04:11:47.411860 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.411835    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 04:11:47.412981 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.412022    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 04:11:47.412981 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.412182    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.414285 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.414254    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.415734 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.415691    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 04:11:47.415821 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.415769    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 04:11:47.415878 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.415845    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 04:11:47.416031 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.416009    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cbcbp\""
Jul 28 04:11:47.416313 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.416294    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 04:11:47.416787 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.416768    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 04:11:47.417510 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.417287    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 04:11:47.417773 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.417745    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mjpwv\""
Jul 28 04:11:47.418423 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.417916    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 04:11:47.418423 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.418263    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 04:11:47.419534 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.418703    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xgf87"
Jul 28 04:11:47.419534 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.418723    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-279wd\""
Jul 28 04:11:47.420630 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.420606    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bzh7h\""
Jul 28 04:11:47.420710 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.420638    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 04:11:47.421039 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.421023    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:11:47.421138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.421117    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 04:11:47.421226 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:47.421205    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:11:47.422553 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422527    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95552c94-10a3-4446-8b07-fc7afaadc0d4-kubelet-config\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:47.422644 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422571    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/679a6032-e93c-4030-b74b-6c5cdf46b470-os-release\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.422644 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422597    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/679a6032-e93c-4030-b74b-6c5cdf46b470-cni-binary-copy\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.422644 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422619    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-run-openvswitch\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422644    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-cni-bin\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422666    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-etc-kubernetes\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422687    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/679a6032-e93c-4030-b74b-6c5cdf46b470-system-cni-dir\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422720    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7a3204d5-3bc9-420f-b595-e5380a4226e4-ovnkube-script-lib\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422748    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-run-k8s-cni-cncf-io\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422771    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/679a6032-e93c-4030-b74b-6c5cdf46b470-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422793    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be313335-4922-4d50-8479-7220961aa707-iptables-alerter-script\") pod \"iptables-alerter-w4rlp\" (UID: \"be313335-4922-4d50-8479-7220961aa707\") " pod="openshift-network-operator/iptables-alerter-w4rlp"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422844    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422894    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a3204d5-3bc9-420f-b595-e5380a4226e4-ovn-node-metrics-cert\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422926    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95552c94-10a3-4446-8b07-fc7afaadc0d4-dbus\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.422950    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-multus-socket-dir-parent\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423019    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423039    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/679a6032-e93c-4030-b74b-6c5cdf46b470-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423072    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.423118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423108    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-var-lib-openvswitch\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423131    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-cni-netd\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423156    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzvs2\" (UniqueName: \"kubernetes.io/projected/7a3204d5-3bc9-420f-b595-e5380a4226e4-kube-api-access-lzvs2\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423179    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-system-cni-dir\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423201    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-cnibin\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423221    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-systemd-units\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423242    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-slash\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423277    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-log-socket\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423316    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423347    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-multus-cni-dir\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423401    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-run-netns\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423425    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-etc-selinux\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423448    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423472    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5aaea95b-0d89-465d-a928-e55805bffaa8-cni-binary-copy\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423506    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-modprobe-d\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423529    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-socket-dir\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.423901 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423552    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-device-dir\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423585    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5aaea95b-0d89-465d-a928-e55805bffaa8-multus-daemon-config\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423618    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-run-ovn\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423640    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-var-lib-kubelet\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423657    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-hostroot\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423678    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7a3204d5-3bc9-420f-b595-e5380a4226e4-ovnkube-config\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423692    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-os-release\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423712    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-var-lib-cni-multus\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423740    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2385840-55b1-4920-aa89-ec64f34697f6-host\") pod \"node-ca-27l4c\" (UID: \"d2385840-55b1-4920-aa89-ec64f34697f6\") " pod="openshift-image-registry/node-ca-27l4c"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423753    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2385840-55b1-4920-aa89-ec64f34697f6-serviceca\") pod \"node-ca-27l4c\" (UID: \"d2385840-55b1-4920-aa89-ec64f34697f6\") " pod="openshift-image-registry/node-ca-27l4c"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423769    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2psk\" (UniqueName: \"kubernetes.io/projected/d2385840-55b1-4920-aa89-ec64f34697f6-kube-api-access-f2psk\") pod \"node-ca-27l4c\" (UID: \"d2385840-55b1-4920-aa89-ec64f34697f6\") " pod="openshift-image-registry/node-ca-27l4c"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423789    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-sysctl-d\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423815    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-lib-modules\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423842    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-run-systemd\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423857    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-var-lib-cni-bin\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423879    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/679a6032-e93c-4030-b74b-6c5cdf46b470-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423897    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-etc-openvswitch\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.424625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423912    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-run-multus-certs\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423929    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/679a6032-e93c-4030-b74b-6c5cdf46b470-cnibin\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.423949    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzh24\" (UniqueName: \"kubernetes.io/projected/679a6032-e93c-4030-b74b-6c5cdf46b470-kube-api-access-bzh24\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424027    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-sysconfig\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424062    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-sysctl-conf\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424075    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-run\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424092    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-sys\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424097    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b47jl"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424121    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-host\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424139    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lwt9\" (UniqueName: \"kubernetes.io/projected/ffa6e27a-0769-421d-a24d-3a9138bec37f-kube-api-access-2lwt9\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424157    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-systemd\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424179    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-var-lib-kubelet\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424200    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6n5w\" (UniqueName: \"kubernetes.io/projected/be313335-4922-4d50-8479-7220961aa707-kube-api-access-h6n5w\") pod \"iptables-alerter-w4rlp\" (UID: \"be313335-4922-4d50-8479-7220961aa707\") " pod="openshift-network-operator/iptables-alerter-w4rlp"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424221    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-tmp\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424241    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmdlg\" (UniqueName: \"kubernetes.io/projected/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-kube-api-access-hmdlg\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424263    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-sys-fs\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424291    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-multus-conf-dir\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.425480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424314    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:47.426283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424345    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-tuned\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.426283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424391    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be313335-4922-4d50-8479-7220961aa707-host-slash\") pod \"iptables-alerter-w4rlp\" (UID: \"be313335-4922-4d50-8479-7220961aa707\") " pod="openshift-network-operator/iptables-alerter-w4rlp"
Jul 28 04:11:47.426283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424417    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgt6v\" (UniqueName: \"kubernetes.io/projected/5aaea95b-0d89-465d-a928-e55805bffaa8-kube-api-access-kgt6v\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.426283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424459    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5ww4\" (UniqueName: \"kubernetes.io/projected/0de07d28-1d49-4736-ac9f-396aff4bfd88-kube-api-access-d5ww4\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.426283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424493    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-kubelet\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.426283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424550    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-node-log\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.426283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424643    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7a3204d5-3bc9-420f-b595-e5380a4226e4-env-overrides\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.426283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424679    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-kubernetes\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.426283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424703    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-registration-dir\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.426283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.424736    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-run-netns\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.426283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.426219    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 04:11:47.426832 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.426568    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 04:11:47.426887 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.426856    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kzlf2\""
Jul 28 04:11:47.440885 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.440864    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xc7j8"
Jul 28 04:11:47.450147 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.450125    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xc7j8"
Jul 28 04:11:47.513099 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.513079    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 04:11:47.518125 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.518100    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 04:11:47.525347 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.525325    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-sysctl-conf\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.525448 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.525374    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-run\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.525448 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.525400    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-sys\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.525448 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.525422    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-host\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.525590 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.525447    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lwt9\" (UniqueName: \"kubernetes.io/projected/ffa6e27a-0769-421d-a24d-3a9138bec37f-kube-api-access-2lwt9\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:47.525590 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.525472    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-systemd\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.525590 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.525492    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-sysctl-conf\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.525590 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.525493    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-var-lib-kubelet\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.525590 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.525534    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6n5w\" (UniqueName: \"kubernetes.io/projected/be313335-4922-4d50-8479-7220961aa707-kube-api-access-h6n5w\") pod \"iptables-alerter-w4rlp\" (UID: \"be313335-4922-4d50-8479-7220961aa707\") " pod="openshift-network-operator/iptables-alerter-w4rlp"
Jul 28 04:11:47.525590 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.525552    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-var-lib-kubelet\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.525841 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.525796    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-tmp\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.525973 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.525928    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-run\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.525973 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.525946    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-host\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.526089 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526025    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-systemd\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.526089 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526080    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-sys\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.526533 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526294    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmdlg\" (UniqueName: \"kubernetes.io/projected/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-kube-api-access-hmdlg\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.526533 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526345    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-sys-fs\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.526533 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526397    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-multus-conf-dir\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.526533 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526426    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:47.526533 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526454    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-tuned\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.526533 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526479    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be313335-4922-4d50-8479-7220961aa707-host-slash\") pod \"iptables-alerter-w4rlp\" (UID: \"be313335-4922-4d50-8479-7220961aa707\") " pod="openshift-network-operator/iptables-alerter-w4rlp"
Jul 28 04:11:47.526533 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526508    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgt6v\" (UniqueName: \"kubernetes.io/projected/5aaea95b-0d89-465d-a928-e55805bffaa8-kube-api-access-kgt6v\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.526875 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526530    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-multus-conf-dir\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.526875 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526539    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5ww4\" (UniqueName: \"kubernetes.io/projected/0de07d28-1d49-4736-ac9f-396aff4bfd88-kube-api-access-d5ww4\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.526875 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526625    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-kubelet\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.526875 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526663    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-node-log\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.526875 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526335    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 04:11:47.527135 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526999    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be313335-4922-4d50-8479-7220961aa707-host-slash\") pod \"iptables-alerter-w4rlp\" (UID: \"be313335-4922-4d50-8479-7220961aa707\") " pod="openshift-network-operator/iptables-alerter-w4rlp"
Jul 28 04:11:47.527289 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:47.527242    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:11:47.527383 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527277    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-kubelet\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.527383 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:47.527343    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs podName:ffa6e27a-0769-421d-a24d-3a9138bec37f nodeName:}" failed. No retries permitted until 2026-07-28 04:11:48.027297209 +0000 UTC m=+2.969136925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs") pod "network-metrics-daemon-npqcf" (UID: "ffa6e27a-0769-421d-a24d-3a9138bec37f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:11:47.527496 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527382    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-node-log\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.527496 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527392    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7a3204d5-3bc9-420f-b595-e5380a4226e4-env-overrides\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.527496 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527433    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad63d3ab-011e-4dd2-ba86-f57c268eefcd-agent-certs\") pod \"konnectivity-agent-b47jl\" (UID: \"ad63d3ab-011e-4dd2-ba86-f57c268eefcd\") " pod="kube-system/konnectivity-agent-b47jl"
Jul 28 04:11:47.527496 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527463    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ec166b8-6374-4e81-9a43-65a6fe177e7a-hosts-file\") pod \"node-resolver-xgf87\" (UID: \"3ec166b8-6374-4e81-9a43-65a6fe177e7a\") " pod="openshift-dns/node-resolver-xgf87"
Jul 28 04:11:47.527685 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527498    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-kubernetes\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.527685 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527561    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-kubernetes\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.527685 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527568    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-registration-dir\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.527685 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527613    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-run-netns\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.527879 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527778    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ec166b8-6374-4e81-9a43-65a6fe177e7a-tmp-dir\") pod \"node-resolver-xgf87\" (UID: \"3ec166b8-6374-4e81-9a43-65a6fe177e7a\") " pod="openshift-dns/node-resolver-xgf87"
Jul 28 04:11:47.527879 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527836    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95552c94-10a3-4446-8b07-fc7afaadc0d4-kubelet-config\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:47.527980 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527884    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-run-netns\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.527980 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527912    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95552c94-10a3-4446-8b07-fc7afaadc0d4-kubelet-config\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:47.527980 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527920    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/679a6032-e93c-4030-b74b-6c5cdf46b470-os-release\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.528127 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.527992    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/679a6032-e93c-4030-b74b-6c5cdf46b470-os-release\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.528127 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528033    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/679a6032-e93c-4030-b74b-6c5cdf46b470-cni-binary-copy\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.528127 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528066    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7a3204d5-3bc9-420f-b595-e5380a4226e4-env-overrides\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.528127 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528090    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-run-openvswitch\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.528330 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528168    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-registration-dir\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.528330 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528237    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-cni-bin\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.528330 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-etc-kubernetes\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.528330 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528298    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/679a6032-e93c-4030-b74b-6c5cdf46b470-system-cni-dir\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.528551 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528337    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7a3204d5-3bc9-420f-b595-e5380a4226e4-ovnkube-script-lib\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.528551 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528385    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-run-k8s-cni-cncf-io\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.528551 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528441    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/679a6032-e93c-4030-b74b-6c5cdf46b470-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.528551 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528480    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be313335-4922-4d50-8479-7220961aa707-iptables-alerter-script\") pod \"iptables-alerter-w4rlp\" (UID: \"be313335-4922-4d50-8479-7220961aa707\") " pod="openshift-network-operator/iptables-alerter-w4rlp"
Jul 28 04:11:47.528551 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528516    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.528551 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528543    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a3204d5-3bc9-420f-b595-e5380a4226e4-ovn-node-metrics-cert\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.528829 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528573    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95552c94-10a3-4446-8b07-fc7afaadc0d4-dbus\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:47.528829 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528604    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-multus-socket-dir-parent\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.528829 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528629    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/679a6032-e93c-4030-b74b-6c5cdf46b470-cni-binary-copy\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.528829 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528641    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/679a6032-e93c-4030-b74b-6c5cdf46b470-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.528829 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528674    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.528829 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528696    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-run-openvswitch\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.528829 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528701    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-var-lib-openvswitch\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.528829 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528747    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-cni-netd\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.528829 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528768    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-run-k8s-cni-cncf-io\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.528829 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528782    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzvs2\" (UniqueName: \"kubernetes.io/projected/7a3204d5-3bc9-420f-b595-e5380a4226e4-kube-api-access-lzvs2\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.528829 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528812    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-system-cni-dir\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528852    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-cnibin\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528885    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-systemd-units\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528916    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-slash\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528946    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-log-socket\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528969    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.528995    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-multus-cni-dir\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529021    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-run-netns\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529046    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-etc-selinux\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529076    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529112    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24wnf\" (UniqueName: \"kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf\") pod \"network-check-target-5rg2c\" (UID: \"c52368c0-8140-4957-b423-dcd1dead91b5\") " pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529145    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5aaea95b-0d89-465d-a928-e55805bffaa8-cni-binary-copy\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-modprobe-d\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529197    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-socket-dir\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529226    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-device-dir\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529251    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/679a6032-e93c-4030-b74b-6c5cdf46b470-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbtkg\" (UniqueName: \"kubernetes.io/projected/3ec166b8-6374-4e81-9a43-65a6fe177e7a-kube-api-access-lbtkg\") pod \"node-resolver-xgf87\" (UID: \"3ec166b8-6374-4e81-9a43-65a6fe177e7a\") " pod="openshift-dns/node-resolver-xgf87"
Jul 28 04:11:47.529334 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529320    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-var-lib-openvswitch\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.530138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529389    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.530138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529394    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-cni-netd\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.530138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529488    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-cnibin\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.530138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529681    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95552c94-10a3-4446-8b07-fc7afaadc0d4-dbus\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:47.530138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529751    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-multus-socket-dir-parent\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.530138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.526629    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-sys-fs\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.530138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529897    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/679a6032-e93c-4030-b74b-6c5cdf46b470-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.530138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529929    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5aaea95b-0d89-465d-a928-e55805bffaa8-multus-daemon-config\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.530138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.529980    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-run-ovn\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.530138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530077    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.530138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530089    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-system-cni-dir\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.530138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530130    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad63d3ab-011e-4dd2-ba86-f57c268eefcd-konnectivity-ca\") pod \"konnectivity-agent-b47jl\" (UID: \"ad63d3ab-011e-4dd2-ba86-f57c268eefcd\") " pod="kube-system/konnectivity-agent-b47jl"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530162    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-etc-kubernetes\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530174    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-var-lib-kubelet\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530206    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-hostroot\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530215    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-run-ovn\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530235    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7a3204d5-3bc9-420f-b595-e5380a4226e4-ovnkube-config\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530286    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-os-release\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530319    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-var-lib-cni-multus\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530349    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2385840-55b1-4920-aa89-ec64f34697f6-host\") pod \"node-ca-27l4c\" (UID: \"d2385840-55b1-4920-aa89-ec64f34697f6\") " pod="openshift-image-registry/node-ca-27l4c"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530402    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2385840-55b1-4920-aa89-ec64f34697f6-serviceca\") pod \"node-ca-27l4c\" (UID: \"d2385840-55b1-4920-aa89-ec64f34697f6\") " pod="openshift-image-registry/node-ca-27l4c"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530433    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2psk\" (UniqueName: \"kubernetes.io/projected/d2385840-55b1-4920-aa89-ec64f34697f6-kube-api-access-f2psk\") pod \"node-ca-27l4c\" (UID: \"d2385840-55b1-4920-aa89-ec64f34697f6\") " pod="openshift-image-registry/node-ca-27l4c"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530465    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-sysctl-d\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530493    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-lib-modules\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530505    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be313335-4922-4d50-8479-7220961aa707-iptables-alerter-script\") pod \"iptables-alerter-w4rlp\" (UID: \"be313335-4922-4d50-8479-7220961aa707\") " pod="openshift-network-operator/iptables-alerter-w4rlp"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530537    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-run-systemd\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530569    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-var-lib-cni-bin\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530596    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/679a6032-e93c-4030-b74b-6c5cdf46b470-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530617    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-tuned\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.530724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530628    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-etc-openvswitch\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530663    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-run-multus-certs\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530695    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/679a6032-e93c-4030-b74b-6c5cdf46b470-cnibin\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530724    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzh24\" (UniqueName: \"kubernetes.io/projected/679a6032-e93c-4030-b74b-6c5cdf46b470-kube-api-access-bzh24\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530761    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-sysconfig\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530799    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-log-socket\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530807    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5aaea95b-0d89-465d-a928-e55805bffaa8-multus-daemon-config\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530876    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-systemd-units\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530893    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-sysconfig\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530954    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-slash\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.530971    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-var-lib-kubelet\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.531099    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-hostroot\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.531347    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-os-release\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.531353    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7a3204d5-3bc9-420f-b595-e5380a4226e4-ovnkube-script-lib\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.531555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.531442    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2385840-55b1-4920-aa89-ec64f34697f6-host\") pod \"node-ca-27l4c\" (UID: \"d2385840-55b1-4920-aa89-ec64f34697f6\") " pod="openshift-image-registry/node-ca-27l4c"
Jul 28 04:11:47.532205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.531719    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/679a6032-e93c-4030-b74b-6c5cdf46b470-system-cni-dir\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.532205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.531784    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-etc-selinux\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.532205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.531786    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2385840-55b1-4920-aa89-ec64f34697f6-serviceca\") pod \"node-ca-27l4c\" (UID: \"d2385840-55b1-4920-aa89-ec64f34697f6\") " pod="openshift-image-registry/node-ca-27l4c"
Jul 28 04:11:47.532205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.531839    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-sysctl-d\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.532205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.531865    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-multus-cni-dir\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.532205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.531921    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-run-netns\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.532205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.531940    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-lib-modules\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.532205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.531992    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-var-lib-cni-bin\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.532205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.532067    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-run-systemd\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.532205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.532137    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/679a6032-e93c-4030-b74b-6c5cdf46b470-cnibin\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.532699 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.532333    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7a3204d5-3bc9-420f-b595-e5380a4226e4-ovnkube-config\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.532699 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.532496    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-tmp\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.532699 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.532560    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-run-multus-certs\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.532699 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.532572    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5aaea95b-0d89-465d-a928-e55805bffaa8-cni-binary-copy\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.532699 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.532598    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-socket-dir\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.532699 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:47.532665    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 04:11:47.532973 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.532711    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-etc-modprobe-d\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.532973 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:47.532740    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret podName:95552c94-10a3-4446-8b07-fc7afaadc0d4 nodeName:}" failed. No retries permitted until 2026-07-28 04:11:48.032720977 +0000 UTC m=+2.974560690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret") pod "global-pull-secret-syncer-wgdm4" (UID: "95552c94-10a3-4446-8b07-fc7afaadc0d4") : object "kube-system"/"original-pull-secret" not registered
Jul 28 04:11:47.532973 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.532770    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0de07d28-1d49-4736-ac9f-396aff4bfd88-device-dir\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.532973 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.532663    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5aaea95b-0d89-465d-a928-e55805bffaa8-host-var-lib-cni-multus\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.532973 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.532801    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-cni-bin\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.532973 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.532835    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-etc-openvswitch\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.533273 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.533108    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/679a6032-e93c-4030-b74b-6c5cdf46b470-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.533401 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.533382    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a3204d5-3bc9-420f-b595-e5380a4226e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.535259 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.535165    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6n5w\" (UniqueName: \"kubernetes.io/projected/be313335-4922-4d50-8479-7220961aa707-kube-api-access-h6n5w\") pod \"iptables-alerter-w4rlp\" (UID: \"be313335-4922-4d50-8479-7220961aa707\") " pod="openshift-network-operator/iptables-alerter-w4rlp"
Jul 28 04:11:47.535554 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.535529    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a3204d5-3bc9-420f-b595-e5380a4226e4-ovn-node-metrics-cert\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.535656 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.535605    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5ww4\" (UniqueName: \"kubernetes.io/projected/0de07d28-1d49-4736-ac9f-396aff4bfd88-kube-api-access-d5ww4\") pod \"aws-ebs-csi-driver-node-sx2rk\" (UID: \"0de07d28-1d49-4736-ac9f-396aff4bfd88\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.537110 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.537088    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lwt9\" (UniqueName: \"kubernetes.io/projected/ffa6e27a-0769-421d-a24d-3a9138bec37f-kube-api-access-2lwt9\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:47.538498 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.538476    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgt6v\" (UniqueName: \"kubernetes.io/projected/5aaea95b-0d89-465d-a928-e55805bffaa8-kube-api-access-kgt6v\") pod \"multus-zl45v\" (UID: \"5aaea95b-0d89-465d-a928-e55805bffaa8\") " pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.538597 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.538547    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmdlg\" (UniqueName: \"kubernetes.io/projected/d3459f97-1b6e-4f6d-b92f-1081c0207e5a-kube-api-access-hmdlg\") pod \"tuned-zq2l4\" (UID: \"d3459f97-1b6e-4f6d-b92f-1081c0207e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.538865 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.538845    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2psk\" (UniqueName: \"kubernetes.io/projected/d2385840-55b1-4920-aa89-ec64f34697f6-kube-api-access-f2psk\") pod \"node-ca-27l4c\" (UID: \"d2385840-55b1-4920-aa89-ec64f34697f6\") " pod="openshift-image-registry/node-ca-27l4c"
Jul 28 04:11:47.539739 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.539717    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzh24\" (UniqueName: \"kubernetes.io/projected/679a6032-e93c-4030-b74b-6c5cdf46b470-kube-api-access-bzh24\") pod \"multus-additional-cni-plugins-mrsls\" (UID: \"679a6032-e93c-4030-b74b-6c5cdf46b470\") " pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.540333 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.540307    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzvs2\" (UniqueName: \"kubernetes.io/projected/7a3204d5-3bc9-420f-b595-e5380a4226e4-kube-api-access-lzvs2\") pod \"ovnkube-node-tb5v6\" (UID: \"7a3204d5-3bc9-420f-b595-e5380a4226e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.631611 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.631582    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24wnf\" (UniqueName: \"kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf\") pod \"network-check-target-5rg2c\" (UID: \"c52368c0-8140-4957-b423-dcd1dead91b5\") " pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:11:47.631750 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.631623    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbtkg\" (UniqueName: \"kubernetes.io/projected/3ec166b8-6374-4e81-9a43-65a6fe177e7a-kube-api-access-lbtkg\") pod \"node-resolver-xgf87\" (UID: \"3ec166b8-6374-4e81-9a43-65a6fe177e7a\") " pod="openshift-dns/node-resolver-xgf87"
Jul 28 04:11:47.631750 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.631649    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad63d3ab-011e-4dd2-ba86-f57c268eefcd-konnectivity-ca\") pod \"konnectivity-agent-b47jl\" (UID: \"ad63d3ab-011e-4dd2-ba86-f57c268eefcd\") " pod="kube-system/konnectivity-agent-b47jl"
Jul 28 04:11:47.631874 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.631830    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad63d3ab-011e-4dd2-ba86-f57c268eefcd-agent-certs\") pod \"konnectivity-agent-b47jl\" (UID: \"ad63d3ab-011e-4dd2-ba86-f57c268eefcd\") " pod="kube-system/konnectivity-agent-b47jl"
Jul 28 04:11:47.631930 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.631878    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ec166b8-6374-4e81-9a43-65a6fe177e7a-hosts-file\") pod \"node-resolver-xgf87\" (UID: \"3ec166b8-6374-4e81-9a43-65a6fe177e7a\") " pod="openshift-dns/node-resolver-xgf87"
Jul 28 04:11:47.631930 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.631908    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ec166b8-6374-4e81-9a43-65a6fe177e7a-tmp-dir\") pod \"node-resolver-xgf87\" (UID: \"3ec166b8-6374-4e81-9a43-65a6fe177e7a\") " pod="openshift-dns/node-resolver-xgf87"
Jul 28 04:11:47.632024 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.631942    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ec166b8-6374-4e81-9a43-65a6fe177e7a-hosts-file\") pod \"node-resolver-xgf87\" (UID: \"3ec166b8-6374-4e81-9a43-65a6fe177e7a\") " pod="openshift-dns/node-resolver-xgf87"
Jul 28 04:11:47.632207 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.632189    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ec166b8-6374-4e81-9a43-65a6fe177e7a-tmp-dir\") pod \"node-resolver-xgf87\" (UID: \"3ec166b8-6374-4e81-9a43-65a6fe177e7a\") " pod="openshift-dns/node-resolver-xgf87"
Jul 28 04:11:47.632286 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.632195    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad63d3ab-011e-4dd2-ba86-f57c268eefcd-konnectivity-ca\") pod \"konnectivity-agent-b47jl\" (UID: \"ad63d3ab-011e-4dd2-ba86-f57c268eefcd\") " pod="kube-system/konnectivity-agent-b47jl"
Jul 28 04:11:47.634207 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.634188    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad63d3ab-011e-4dd2-ba86-f57c268eefcd-agent-certs\") pod \"konnectivity-agent-b47jl\" (UID: \"ad63d3ab-011e-4dd2-ba86-f57c268eefcd\") " pod="kube-system/konnectivity-agent-b47jl"
Jul 28 04:11:47.638741 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:47.638720    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 04:11:47.638816 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:47.638748    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 04:11:47.638816 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:47.638761    2574 projected.go:196] Error preparing data for projected volume kube-api-access-24wnf for pod openshift-network-diagnostics/network-check-target-5rg2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:11:47.638906 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:47.638819    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf podName:c52368c0-8140-4957-b423-dcd1dead91b5 nodeName:}" failed. No retries permitted until 2026-07-28 04:11:48.138806917 +0000 UTC m=+3.080646615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-24wnf" (UniqueName: "kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf") pod "network-check-target-5rg2c" (UID: "c52368c0-8140-4957-b423-dcd1dead91b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:11:47.640736 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.640719    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbtkg\" (UniqueName: \"kubernetes.io/projected/3ec166b8-6374-4e81-9a43-65a6fe177e7a-kube-api-access-lbtkg\") pod \"node-resolver-xgf87\" (UID: \"3ec166b8-6374-4e81-9a43-65a6fe177e7a\") " pod="openshift-dns/node-resolver-xgf87"
Jul 28 04:11:47.715881 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.715827    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:11:47.730760 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.730740    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zl45v"
Jul 28 04:11:47.739146 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.739116    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zq2l4"
Jul 28 04:11:47.745444 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.745425    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-27l4c"
Jul 28 04:11:47.759771 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.759752    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w4rlp"
Jul 28 04:11:47.768267 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.768247    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk"
Jul 28 04:11:47.775694 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.775672    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mrsls"
Jul 28 04:11:47.782176 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.782157    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xgf87"
Jul 28 04:11:47.788575 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.788556    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b47jl"
Jul 28 04:11:47.878999 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:47.878975    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 04:11:48.036027 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.036000    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:48.036204 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.036052    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:48.036204 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:48.036152    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:11:48.036316 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:48.036218    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs podName:ffa6e27a-0769-421d-a24d-3a9138bec37f nodeName:}" failed. No retries permitted until 2026-07-28 04:11:49.036201652 +0000 UTC m=+3.978041364 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs") pod "network-metrics-daemon-npqcf" (UID: "ffa6e27a-0769-421d-a24d-3a9138bec37f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:11:48.036316 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:48.036153    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 04:11:48.036316 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:48.036290    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret podName:95552c94-10a3-4446-8b07-fc7afaadc0d4 nodeName:}" failed. No retries permitted until 2026-07-28 04:11:49.036277483 +0000 UTC m=+3.978117185 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret") pod "global-pull-secret-syncer-wgdm4" (UID: "95552c94-10a3-4446-8b07-fc7afaadc0d4") : object "kube-system"/"original-pull-secret" not registered
Jul 28 04:11:48.237681 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.237652    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24wnf\" (UniqueName: \"kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf\") pod \"network-check-target-5rg2c\" (UID: \"c52368c0-8140-4957-b423-dcd1dead91b5\") " pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:11:48.237849 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:48.237830    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 04:11:48.237905 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:48.237858    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 04:11:48.237905 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:48.237871    2574 projected.go:196] Error preparing data for projected volume kube-api-access-24wnf for pod openshift-network-diagnostics/network-check-target-5rg2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:11:48.237993 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:48.237936    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf podName:c52368c0-8140-4957-b423-dcd1dead91b5 nodeName:}" failed. No retries permitted until 2026-07-28 04:11:49.237917367 +0000 UTC m=+4.179757068 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-24wnf" (UniqueName: "kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf") pod "network-check-target-5rg2c" (UID: "c52368c0-8140-4957-b423-dcd1dead91b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:11:48.452096 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:48.452074    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0de07d28_1d49_4736_ac9f_396aff4bfd88.slice/crio-5d0a1e0d705332c05e7df525d60e8bf45a5a471a2b1922f07ed6cfcda0047d42 WatchSource:0}: Error finding container 5d0a1e0d705332c05e7df525d60e8bf45a5a471a2b1922f07ed6cfcda0047d42: Status 404 returned error can't find the container with id 5d0a1e0d705332c05e7df525d60e8bf45a5a471a2b1922f07ed6cfcda0047d42
Jul 28 04:11:48.452565 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.452454    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 04:06:47 +0000 UTC" deadline="2028-02-21 06:36:17.311196676 +0000 UTC"
Jul 28 04:11:48.452565 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.452483    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13754h24m28.858717923s"
Jul 28 04:11:48.453194 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:48.453106    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2385840_55b1_4920_aa89_ec64f34697f6.slice/crio-70608ce4b1cf68f148fe21b54fb178e6c41047d01f6c87c807c2cb7b8713b475 WatchSource:0}: Error finding container 70608ce4b1cf68f148fe21b54fb178e6c41047d01f6c87c807c2cb7b8713b475: Status 404 returned error can't find the container with id 70608ce4b1cf68f148fe21b54fb178e6c41047d01f6c87c807c2cb7b8713b475
Jul 28 04:11:48.454084 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:48.453953    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe313335_4922_4d50_8479_7220961aa707.slice/crio-ffb693bbf4e6985bedc688f667a0bd562792d2f6fb7794eddd8d221b8efd4239 WatchSource:0}: Error finding container ffb693bbf4e6985bedc688f667a0bd562792d2f6fb7794eddd8d221b8efd4239: Status 404 returned error can't find the container with id ffb693bbf4e6985bedc688f667a0bd562792d2f6fb7794eddd8d221b8efd4239
Jul 28 04:11:48.454712 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:48.454688    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3204d5_3bc9_420f_b595_e5380a4226e4.slice/crio-98c8b57c649a96bb87034202dc15d0aece0055c99a25e69acb5b5ab46cece9b4 WatchSource:0}: Error finding container 98c8b57c649a96bb87034202dc15d0aece0055c99a25e69acb5b5ab46cece9b4: Status 404 returned error can't find the container with id 98c8b57c649a96bb87034202dc15d0aece0055c99a25e69acb5b5ab46cece9b4
Jul 28 04:11:48.477412 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:48.477393    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aaea95b_0d89_465d_a928_e55805bffaa8.slice/crio-df9e0273054deca46a5b478b498ad7420bcb761bd27e5a05ea381858b0f5089b WatchSource:0}: Error finding container df9e0273054deca46a5b478b498ad7420bcb761bd27e5a05ea381858b0f5089b: Status 404 returned error can't find the container with id df9e0273054deca46a5b478b498ad7420bcb761bd27e5a05ea381858b0f5089b
Jul 28 04:11:48.478025 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:48.478004    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ec166b8_6374_4e81_9a43_65a6fe177e7a.slice/crio-14d16b93c4988577e2fbc0ba64514a970de67da05a968650f0a62d4c26cf0a46 WatchSource:0}: Error finding container 14d16b93c4988577e2fbc0ba64514a970de67da05a968650f0a62d4c26cf0a46: Status 404 returned error can't find the container with id 14d16b93c4988577e2fbc0ba64514a970de67da05a968650f0a62d4c26cf0a46
Jul 28 04:11:48.479698 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:48.479677    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod679a6032_e93c_4030_b74b_6c5cdf46b470.slice/crio-326b5228e696b16bf4de10754d1d47175bbbde22ccf1894370290b0b072e6060 WatchSource:0}: Error finding container 326b5228e696b16bf4de10754d1d47175bbbde22ccf1894370290b0b072e6060: Status 404 returned error can't find the container with id 326b5228e696b16bf4de10754d1d47175bbbde22ccf1894370290b0b072e6060
Jul 28 04:11:48.481199 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:48.480692    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad63d3ab_011e_4dd2_ba86_f57c268eefcd.slice/crio-ade485f7e2446191466c723bf254012259c57660190a209cdf1aa5106a744552 WatchSource:0}: Error finding container ade485f7e2446191466c723bf254012259c57660190a209cdf1aa5106a744552: Status 404 returned error can't find the container with id ade485f7e2446191466c723bf254012259c57660190a209cdf1aa5106a744552
Jul 28 04:11:48.481599 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:11:48.481579    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3459f97_1b6e_4f6d_b92f_1081c0207e5a.slice/crio-c1590e6c533e2c0f342c4d1ebd3de119ca3f9d84cc6b20f9efc06d448f62131c WatchSource:0}: Error finding container c1590e6c533e2c0f342c4d1ebd3de119ca3f9d84cc6b20f9efc06d448f62131c: Status 404 returned error can't find the container with id c1590e6c533e2c0f342c4d1ebd3de119ca3f9d84cc6b20f9efc06d448f62131c
Jul 28 04:11:48.614898 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.614738    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:48.615002 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:48.614983    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:11:48.624553 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.623591    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-204.ec2.internal" event={"ID":"6db9e0d726c4428a7173af2b853a8bd3","Type":"ContainerStarted","Data":"bd3179fb1cf27e906c1a3fb9cb15fdeca8777e9aa0f19a0765b43e58ccc37c7c"}
Jul 28 04:11:48.626229 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.626115    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zq2l4" event={"ID":"d3459f97-1b6e-4f6d-b92f-1081c0207e5a","Type":"ContainerStarted","Data":"c1590e6c533e2c0f342c4d1ebd3de119ca3f9d84cc6b20f9efc06d448f62131c"}
Jul 28 04:11:48.627486 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.627465    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrsls" event={"ID":"679a6032-e93c-4030-b74b-6c5cdf46b470","Type":"ContainerStarted","Data":"326b5228e696b16bf4de10754d1d47175bbbde22ccf1894370290b0b072e6060"}
Jul 28 04:11:48.628502 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.628481    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xgf87" event={"ID":"3ec166b8-6374-4e81-9a43-65a6fe177e7a","Type":"ContainerStarted","Data":"14d16b93c4988577e2fbc0ba64514a970de67da05a968650f0a62d4c26cf0a46"}
Jul 28 04:11:48.629577 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.629553    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zl45v" event={"ID":"5aaea95b-0d89-465d-a928-e55805bffaa8","Type":"ContainerStarted","Data":"df9e0273054deca46a5b478b498ad7420bcb761bd27e5a05ea381858b0f5089b"}
Jul 28 04:11:48.630631 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.630612    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w4rlp" event={"ID":"be313335-4922-4d50-8479-7220961aa707","Type":"ContainerStarted","Data":"ffb693bbf4e6985bedc688f667a0bd562792d2f6fb7794eddd8d221b8efd4239"}
Jul 28 04:11:48.631675 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.631652    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-27l4c" event={"ID":"d2385840-55b1-4920-aa89-ec64f34697f6","Type":"ContainerStarted","Data":"70608ce4b1cf68f148fe21b54fb178e6c41047d01f6c87c807c2cb7b8713b475"}
Jul 28 04:11:48.632684 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.632665    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b47jl" event={"ID":"ad63d3ab-011e-4dd2-ba86-f57c268eefcd","Type":"ContainerStarted","Data":"ade485f7e2446191466c723bf254012259c57660190a209cdf1aa5106a744552"}
Jul 28 04:11:48.633692 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.633674    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6" event={"ID":"7a3204d5-3bc9-420f-b595-e5380a4226e4","Type":"ContainerStarted","Data":"98c8b57c649a96bb87034202dc15d0aece0055c99a25e69acb5b5ab46cece9b4"}
Jul 28 04:11:48.634600 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.634581    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk" event={"ID":"0de07d28-1d49-4736-ac9f-396aff4bfd88","Type":"ContainerStarted","Data":"5d0a1e0d705332c05e7df525d60e8bf45a5a471a2b1922f07ed6cfcda0047d42"}
Jul 28 04:11:48.636293 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:48.636249    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-204.ec2.internal" podStartSLOduration=2.636232518 podStartE2EDuration="2.636232518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 04:11:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:11:48.63550466 +0000 UTC m=+3.577344379" watchObservedRunningTime="2026-07-28 04:11:48.636232518 +0000 UTC m=+3.578072240"
Jul 28 04:11:49.045785 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:49.045018    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:49.045785 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:49.045082    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:49.045785 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:49.045220    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 04:11:49.045785 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:49.045279    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret podName:95552c94-10a3-4446-8b07-fc7afaadc0d4 nodeName:}" failed. No retries permitted until 2026-07-28 04:11:51.045262 +0000 UTC m=+5.987101705 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret") pod "global-pull-secret-syncer-wgdm4" (UID: "95552c94-10a3-4446-8b07-fc7afaadc0d4") : object "kube-system"/"original-pull-secret" not registered
Jul 28 04:11:49.045785 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:49.045701    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:11:49.045785 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:49.045748    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs podName:ffa6e27a-0769-421d-a24d-3a9138bec37f nodeName:}" failed. No retries permitted until 2026-07-28 04:11:51.045733705 +0000 UTC m=+5.987573410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs") pod "network-metrics-daemon-npqcf" (UID: "ffa6e27a-0769-421d-a24d-3a9138bec37f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:11:49.247686 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:49.247588    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24wnf\" (UniqueName: \"kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf\") pod \"network-check-target-5rg2c\" (UID: \"c52368c0-8140-4957-b423-dcd1dead91b5\") " pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:11:49.247900 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:49.247773    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 04:11:49.247900 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:49.247800    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 04:11:49.247900 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:49.247813    2574 projected.go:196] Error preparing data for projected volume kube-api-access-24wnf for pod openshift-network-diagnostics/network-check-target-5rg2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:11:49.247900 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:49.247870    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf podName:c52368c0-8140-4957-b423-dcd1dead91b5 nodeName:}" failed. No retries permitted until 2026-07-28 04:11:51.247851546 +0000 UTC m=+6.189691250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-24wnf" (UniqueName: "kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf") pod "network-check-target-5rg2c" (UID: "c52368c0-8140-4957-b423-dcd1dead91b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:11:49.453562 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:49.453438    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 04:06:47 +0000 UTC" deadline="2028-02-06 05:58:28.394616757 +0000 UTC"
Jul 28 04:11:49.453562 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:49.453492    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13393h46m38.941129082s"
Jul 28 04:11:49.617061 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:49.616962    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:11:49.617214 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:49.617079    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:11:49.617526 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:49.617510    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:49.617806 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:49.617604    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:11:49.651505 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:49.651436    2574 generic.go:356] "Generic (PLEG): container finished" podID="5e83b5d2069ff1f9446be40c52eae0c3" containerID="0f7ce6ddc5ce6f8a628aa5b27564d9a9125dd20876b34a74962361ced0e0c71f" exitCode=0
Jul 28 04:11:49.651653 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:49.651577    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal" event={"ID":"5e83b5d2069ff1f9446be40c52eae0c3","Type":"ContainerDied","Data":"0f7ce6ddc5ce6f8a628aa5b27564d9a9125dd20876b34a74962361ced0e0c71f"}
Jul 28 04:11:50.614843 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:50.614813    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:50.615291 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:50.614956    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:11:50.680190 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:50.679541    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal" event={"ID":"5e83b5d2069ff1f9446be40c52eae0c3","Type":"ContainerStarted","Data":"45eead4b18ec4de00fcaad65f1cadbfabe4fa87ec9ff2bee39d5a1556a34d129"}
Jul 28 04:11:51.061847 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:51.061808    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:51.062039 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:51.061895    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:51.062039 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:51.061971    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 04:11:51.062039 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:51.061989    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:11:51.062198 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:51.062050    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret podName:95552c94-10a3-4446-8b07-fc7afaadc0d4 nodeName:}" failed. No retries permitted until 2026-07-28 04:11:55.062028605 +0000 UTC m=+10.003868308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret") pod "global-pull-secret-syncer-wgdm4" (UID: "95552c94-10a3-4446-8b07-fc7afaadc0d4") : object "kube-system"/"original-pull-secret" not registered
Jul 28 04:11:51.062198 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:51.062073    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs podName:ffa6e27a-0769-421d-a24d-3a9138bec37f nodeName:}" failed. No retries permitted until 2026-07-28 04:11:55.062062684 +0000 UTC m=+10.003902398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs") pod "network-metrics-daemon-npqcf" (UID: "ffa6e27a-0769-421d-a24d-3a9138bec37f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:11:51.267815 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:51.265941    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24wnf\" (UniqueName: \"kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf\") pod \"network-check-target-5rg2c\" (UID: \"c52368c0-8140-4957-b423-dcd1dead91b5\") " pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:11:51.267815 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:51.266184    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 04:11:51.267815 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:51.266204    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 04:11:51.267815 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:51.266216    2574 projected.go:196] Error preparing data for projected volume kube-api-access-24wnf for pod openshift-network-diagnostics/network-check-target-5rg2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:11:51.267815 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:51.266274    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf podName:c52368c0-8140-4957-b423-dcd1dead91b5 nodeName:}" failed. No retries permitted until 2026-07-28 04:11:55.266255514 +0000 UTC m=+10.208095216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-24wnf" (UniqueName: "kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf") pod "network-check-target-5rg2c" (UID: "c52368c0-8140-4957-b423-dcd1dead91b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:11:51.615225 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:51.615194    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:11:51.615971 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:51.615321    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:11:51.615971 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:51.615199    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:51.615971 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:51.615526    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:11:52.615242 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:52.614734    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:52.615242 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:52.614874    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:11:53.614819 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:53.614369    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:11:53.614819 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:53.614487    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:11:53.614819 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:53.614698    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:53.614819 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:53.614803    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:11:54.614463 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:54.614432    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:54.614848 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:54.614564    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:11:55.095116 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:55.095083    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:55.095267 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:55.095165    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:55.095267 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:55.095228    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 04:11:55.095353 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:55.095280    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:11:55.095353 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:55.095298    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret podName:95552c94-10a3-4446-8b07-fc7afaadc0d4 nodeName:}" failed. No retries permitted until 2026-07-28 04:12:03.095277127 +0000 UTC m=+18.037116828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret") pod "global-pull-secret-syncer-wgdm4" (UID: "95552c94-10a3-4446-8b07-fc7afaadc0d4") : object "kube-system"/"original-pull-secret" not registered
Jul 28 04:11:55.095353 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:55.095329    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs podName:ffa6e27a-0769-421d-a24d-3a9138bec37f nodeName:}" failed. No retries permitted until 2026-07-28 04:12:03.095312943 +0000 UTC m=+18.037152648 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs") pod "network-metrics-daemon-npqcf" (UID: "ffa6e27a-0769-421d-a24d-3a9138bec37f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:11:55.296228 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:55.296194    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24wnf\" (UniqueName: \"kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf\") pod \"network-check-target-5rg2c\" (UID: \"c52368c0-8140-4957-b423-dcd1dead91b5\") " pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:11:55.296422 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:55.296350    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 04:11:55.296422 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:55.296390    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 04:11:55.296422 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:55.296403    2574 projected.go:196] Error preparing data for projected volume kube-api-access-24wnf for pod openshift-network-diagnostics/network-check-target-5rg2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:11:55.296579 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:55.296465    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf podName:c52368c0-8140-4957-b423-dcd1dead91b5 nodeName:}" failed. No retries permitted until 2026-07-28 04:12:03.296445747 +0000 UTC m=+18.238285446 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-24wnf" (UniqueName: "kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf") pod "network-check-target-5rg2c" (UID: "c52368c0-8140-4957-b423-dcd1dead91b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:11:55.615655 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:55.615030    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:11:55.615655 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:55.615132    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:11:55.615655 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:55.615529    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:55.615655 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:55.615620    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:11:56.614950 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:56.614868    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:56.615113 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:56.615007    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:11:57.614918 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:57.614888    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:57.615262 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:57.614895    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:11:57.615262 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:57.614986    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:11:57.615262 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:57.615059    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:11:58.614776 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:58.614743    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:11:58.614946 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:58.614848    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:11:59.614639 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:59.614605    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:11:59.614819 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:11:59.614608    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:11:59.614819 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:59.614716    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:11:59.614949 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:11:59.614817    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:12:00.614252 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:00.614225    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:00.614456 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:00.614346    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:12:01.617914 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:01.617883    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:01.618333 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:01.617883    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:01.618333 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:01.617987    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:12:01.618333 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:01.618043    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:12:02.614193 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:02.614160    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:02.614398 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:02.614283    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:12:03.161143 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:03.161108    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:03.161669 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:03.161191    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:03.161669 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:03.161266    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 04:12:03.161669 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:03.161306    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:12:03.161669 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:03.161326    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret podName:95552c94-10a3-4446-8b07-fc7afaadc0d4 nodeName:}" failed. No retries permitted until 2026-07-28 04:12:19.161309988 +0000 UTC m=+34.103149690 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret") pod "global-pull-secret-syncer-wgdm4" (UID: "95552c94-10a3-4446-8b07-fc7afaadc0d4") : object "kube-system"/"original-pull-secret" not registered
Jul 28 04:12:03.161669 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:03.161346    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs podName:ffa6e27a-0769-421d-a24d-3a9138bec37f nodeName:}" failed. No retries permitted until 2026-07-28 04:12:19.161334813 +0000 UTC m=+34.103174510 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs") pod "network-metrics-daemon-npqcf" (UID: "ffa6e27a-0769-421d-a24d-3a9138bec37f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:12:03.363763 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:03.363735    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24wnf\" (UniqueName: \"kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf\") pod \"network-check-target-5rg2c\" (UID: \"c52368c0-8140-4957-b423-dcd1dead91b5\") " pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:03.363924 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:03.363863    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 04:12:03.363924 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:03.363879    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 04:12:03.363924 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:03.363892    2574 projected.go:196] Error preparing data for projected volume kube-api-access-24wnf for pod openshift-network-diagnostics/network-check-target-5rg2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:12:03.364080 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:03.363947    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf podName:c52368c0-8140-4957-b423-dcd1dead91b5 nodeName:}" failed. No retries permitted until 2026-07-28 04:12:19.363930816 +0000 UTC m=+34.305770532 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-24wnf" (UniqueName: "kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf") pod "network-check-target-5rg2c" (UID: "c52368c0-8140-4957-b423-dcd1dead91b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:12:03.615129 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:03.615093    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:03.615302 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:03.615131    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:03.615302 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:03.615221    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:12:03.615439 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:03.615341    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:12:04.614861 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:04.614827    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:04.615303 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:04.614955    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:12:05.617495 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:05.617463    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:05.617870 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:05.617566    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:12:05.617870 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:05.617617    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:05.617870 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:05.617711    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:12:06.614605 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:06.614578    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:06.614762 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:06.614672    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:12:07.618353 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.617935    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:07.618975 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.617959    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:07.618975 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:07.618422    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:12:07.618975 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:07.618472    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:12:07.720581 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.720550    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b47jl" event={"ID":"ad63d3ab-011e-4dd2-ba86-f57c268eefcd","Type":"ContainerStarted","Data":"0355fb44f82225e640f86ba140adecc2aebb826fed57dc47785307822da7c31b"}
Jul 28 04:12:07.723373 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.723331    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6" event={"ID":"7a3204d5-3bc9-420f-b595-e5380a4226e4","Type":"ContainerStarted","Data":"25c9f65baa0aa8a806149cda5d16eec83d52dce66f9c3addab7420c837d422a5"}
Jul 28 04:12:07.723510 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.723413    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6" event={"ID":"7a3204d5-3bc9-420f-b595-e5380a4226e4","Type":"ContainerStarted","Data":"59b2da207a03cf1513a65008c3302fecce127d0549455de65dfa064a25032e21"}
Jul 28 04:12:07.723510 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.723456    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6" event={"ID":"7a3204d5-3bc9-420f-b595-e5380a4226e4","Type":"ContainerStarted","Data":"61d145046001e33434674e9885e400ae2c7d511fcca5ffd3a1be2e7593ed5d41"}
Jul 28 04:12:07.723510 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.723503    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6" event={"ID":"7a3204d5-3bc9-420f-b595-e5380a4226e4","Type":"ContainerStarted","Data":"8df5bd5bf09ad65404f4c91f59ad0a8782e703a6ac415aab10975d1ceaa7f98b"}
Jul 28 04:12:07.723641 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.723540    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6" event={"ID":"7a3204d5-3bc9-420f-b595-e5380a4226e4","Type":"ContainerStarted","Data":"7b4a0832f602cd6d45e5479d590a5cb002b4b68f862712b775893ec932abf0a0"}
Jul 28 04:12:07.723641 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.723586    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6" event={"ID":"7a3204d5-3bc9-420f-b595-e5380a4226e4","Type":"ContainerStarted","Data":"36b50faf89b6154bff652d572fed64da00c5bda1cc1973f4add796a94a1b5e73"}
Jul 28 04:12:07.724790 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.724773    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk" event={"ID":"0de07d28-1d49-4736-ac9f-396aff4bfd88","Type":"ContainerStarted","Data":"3893770c7f58a4217d53dfef5fa652d43e3ae08ae7ff65db2ed76c1e24c4fc95"}
Jul 28 04:12:07.726104 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.726083    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zq2l4" event={"ID":"d3459f97-1b6e-4f6d-b92f-1081c0207e5a","Type":"ContainerStarted","Data":"67646e906f678223fc262bb5ea31408fff513e35d5ee2e6fc4b69403bf66c9b2"}
Jul 28 04:12:07.727309 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.727285    2574 generic.go:356] "Generic (PLEG): container finished" podID="679a6032-e93c-4030-b74b-6c5cdf46b470" containerID="4d9ade30caccf013b5d15af1699fcbbb3555d0a590dd185af23b341f887e1c49" exitCode=0
Jul 28 04:12:07.727416 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.727322    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrsls" event={"ID":"679a6032-e93c-4030-b74b-6c5cdf46b470","Type":"ContainerDied","Data":"4d9ade30caccf013b5d15af1699fcbbb3555d0a590dd185af23b341f887e1c49"}
Jul 28 04:12:07.728792 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.728759    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xgf87" event={"ID":"3ec166b8-6374-4e81-9a43-65a6fe177e7a","Type":"ContainerStarted","Data":"f7b2e50e88aae1d4aa0e66ac65781ffe5c1fea21a19c94b19cc6391ea16b9700"}
Jul 28 04:12:07.730120 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.730103    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zl45v" event={"ID":"5aaea95b-0d89-465d-a928-e55805bffaa8","Type":"ContainerStarted","Data":"dd0bd2d1c08f3b72238d4f38cd7ea92e72233f3f9e6b88560cfac60285b1939e"}
Jul 28 04:12:07.731321 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.731305    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-27l4c" event={"ID":"d2385840-55b1-4920-aa89-ec64f34697f6","Type":"ContainerStarted","Data":"c0f3c8d65803f66aef85516b634afb506b60538b370e30ad9361754180a2ab1e"}
Jul 28 04:12:07.733154 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.733122    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-204.ec2.internal" podStartSLOduration=21.733111173 podStartE2EDuration="21.733111173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 04:11:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:11:50.694662071 +0000 UTC m=+5.636501792" watchObservedRunningTime="2026-07-28 04:12:07.733111173 +0000 UTC m=+22.674950896"
Jul 28 04:12:07.751161 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.751124    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b47jl" podStartSLOduration=8.757338337 podStartE2EDuration="22.751109113s" totalImagesPullingTime="13.993770776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:12:07.732746932 +0000 UTC m=+22.674586656" watchObservedRunningTime="2026-07-28 04:12:07.751109113 +0000 UTC m=+22.692948834"
Jul 28 04:12:07.764247 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.764217    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xgf87" podStartSLOduration=8.770491114 podStartE2EDuration="22.764203083s" totalImagesPullingTime="13.993711969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 04:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:12:07.764122815 +0000 UTC m=+22.705962535" watchObservedRunningTime="2026-07-28 04:12:07.764203083 +0000 UTC m=+22.706042804"
Jul 28 04:12:07.778119 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.778089    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zl45v" podStartSLOduration=4.197707158 podStartE2EDuration="22.778078776s" totalImagesPullingTime="18.580371618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 04:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:12:07.777949854 +0000 UTC m=+22.719789576" watchObservedRunningTime="2026-07-28 04:12:07.778078776 +0000 UTC m=+22.719918495"
Jul 28 04:12:07.789426 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.789395    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-27l4c" podStartSLOduration=4.475257062 podStartE2EDuration="22.789385187s" totalImagesPullingTime="18.314128125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 04:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:12:07.789025989 +0000 UTC m=+22.730865709" watchObservedRunningTime="2026-07-28 04:12:07.789385187 +0000 UTC m=+22.731224908"
Jul 28 04:12:07.802892 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:07.802860    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zq2l4" podStartSLOduration=4.494486227 podStartE2EDuration="22.802850203s" totalImagesPullingTime="18.308363976s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 04:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:12:07.802693208 +0000 UTC m=+22.744532929" watchObservedRunningTime="2026-07-28 04:12:07.802850203 +0000 UTC m=+22.744689923"
Jul 28 04:12:08.390895 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:08.390872    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 04:12:08.490943 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:08.490819    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T04:12:08.390892566Z","UUID":"ed1d6a18-412a-4f51-990d-50e1de8a92eb","Handler":null,"Name":"","Endpoint":""}
Jul 28 04:12:08.492344 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:08.492315    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 04:12:08.492454 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:08.492349    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 04:12:08.614979 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:08.614916    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:08.615105 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:08.615030    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:12:08.735285 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:08.735255    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w4rlp" event={"ID":"be313335-4922-4d50-8479-7220961aa707","Type":"ContainerStarted","Data":"2e19671078b5fa33f33eeb6211bba0c005c16304ffe5867f3d8235632265cc1d"}
Jul 28 04:12:08.737272 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:08.737228    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk" event={"ID":"0de07d28-1d49-4736-ac9f-396aff4bfd88","Type":"ContainerStarted","Data":"8bddb50f1c856b909af54baf19d57d49b16194d57a988f146e8722aaea549a86"}
Jul 28 04:12:08.747497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:08.747456    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-w4rlp" podStartSLOduration=5.433032696 podStartE2EDuration="23.747441342s" totalImagesPullingTime="18.314408646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 04:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:12:08.747413654 +0000 UTC m=+23.689253376" watchObservedRunningTime="2026-07-28 04:12:08.747441342 +0000 UTC m=+23.689281063"
Jul 28 04:12:09.614570 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:09.614387    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:09.614722 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:09.614437    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:09.614722 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:09.614692    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:12:09.614830 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:09.614737    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:12:09.743084 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:09.743006    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6" event={"ID":"7a3204d5-3bc9-420f-b595-e5380a4226e4","Type":"ContainerStarted","Data":"ac66405abb397c4a4a4bc347d2ea75ddf40e10f115771be5bd4417664841e049"}
Jul 28 04:12:09.745287 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:09.745256    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk" event={"ID":"0de07d28-1d49-4736-ac9f-396aff4bfd88","Type":"ContainerStarted","Data":"bf83224d47501530dc9ac6c6d514024dc726c7fd725f8132a628837d86e4108e"}
Jul 28 04:12:09.759915 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:09.759867    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sx2rk" podStartSLOduration=4.204960345 podStartE2EDuration="24.759848189s" totalImagesPullingTime="20.554887844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 04:11:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:12:09.759798059 +0000 UTC m=+24.701637780" watchObservedRunningTime="2026-07-28 04:12:09.759848189 +0000 UTC m=+24.701687900"
Jul 28 04:12:10.614997 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:10.614968    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:10.615186 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:10.615103    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:12:11.614500 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:11.614473    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:11.614976 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:11.614470    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:11.614976 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:11.614604    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:12:11.614976 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:11.614671    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:12:12.234659 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.234501    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b47jl"
Jul 28 04:12:12.235022 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.235003    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b47jl"
Jul 28 04:12:12.614723 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.614701    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:12.615392 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:12.614816    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:12:12.753719 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.753694    2574 generic.go:356] "Generic (PLEG): container finished" podID="679a6032-e93c-4030-b74b-6c5cdf46b470" containerID="21c2c105704c159b5f7afcc47d0723f9aaede221cdc9d77573de3a4dfe176099" exitCode=0
Jul 28 04:12:12.753858 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.753778    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrsls" event={"ID":"679a6032-e93c-4030-b74b-6c5cdf46b470","Type":"ContainerDied","Data":"21c2c105704c159b5f7afcc47d0723f9aaede221cdc9d77573de3a4dfe176099"}
Jul 28 04:12:12.757246 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.757219    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6" event={"ID":"7a3204d5-3bc9-420f-b595-e5380a4226e4","Type":"ContainerStarted","Data":"79bca6bde9509245352af9c29b35b34dacaae638155280cca3011ce8b31dfdd9"}
Jul 28 04:12:12.757585 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.757544    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:12:12.757711 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.757693    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:12:12.757777 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.757759    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b47jl"
Jul 28 04:12:12.757816 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.757789    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:12:12.758153 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.758136    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b47jl"
Jul 28 04:12:12.772385 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.772351    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:12:12.772523 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.772508    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:12:12.808591 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:12.808555    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6" podStartSLOduration=9.447685861 podStartE2EDuration="27.808543538s" totalImagesPullingTime="18.360857677s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 04:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:12:12.808145952 +0000 UTC m=+27.749985673" watchObservedRunningTime="2026-07-28 04:12:12.808543538 +0000 UTC m=+27.750383258"
Jul 28 04:12:13.617508 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:13.617457    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:13.617508 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:13.617469    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:13.617945 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:13.617549    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:12:13.617945 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:13.617671    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:12:13.970398 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:13.970233    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-npqcf"]
Jul 28 04:12:13.970546 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:13.970534    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:13.970675 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:13.970649    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:12:13.973348 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:13.973326    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wgdm4"]
Jul 28 04:12:13.973493 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:13.973437    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:13.973556 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:13.973534    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:12:13.974149 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:13.974127    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5rg2c"]
Jul 28 04:12:13.974227 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:13.974201    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:13.974337 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:13.974271    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:12:14.763594 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:14.763562    2574 generic.go:356] "Generic (PLEG): container finished" podID="679a6032-e93c-4030-b74b-6c5cdf46b470" containerID="57419ca42f9772d77213c6a95d34d586c9052e8affc2d9e22304ec45c47267e9" exitCode=0
Jul 28 04:12:14.764057 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:14.763646    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrsls" event={"ID":"679a6032-e93c-4030-b74b-6c5cdf46b470","Type":"ContainerDied","Data":"57419ca42f9772d77213c6a95d34d586c9052e8affc2d9e22304ec45c47267e9"}
Jul 28 04:12:15.615864 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:15.615791    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:15.616018 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:15.615886    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:12:15.616018 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:15.615979    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:15.616126 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:15.616105    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:12:15.616181 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:15.616157    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:15.616247 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:15.616230    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:12:15.768071 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:15.768040    2574 generic.go:356] "Generic (PLEG): container finished" podID="679a6032-e93c-4030-b74b-6c5cdf46b470" containerID="48bb7db2bd242281fcdf3a91745c068ed5de34a1f6b37052665160e0821c0b13" exitCode=0
Jul 28 04:12:15.768444 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:15.768091    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrsls" event={"ID":"679a6032-e93c-4030-b74b-6c5cdf46b470","Type":"ContainerDied","Data":"48bb7db2bd242281fcdf3a91745c068ed5de34a1f6b37052665160e0821c0b13"}
Jul 28 04:12:17.615052 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:17.614983    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:17.615052 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:17.615007    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:17.615873 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:17.615108    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:12:17.615873 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:17.615136    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:17.615873 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:17.615332    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:12:17.615873 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:17.615504    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:12:19.186034 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.185994    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:19.186518 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.186065    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:19.186518 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:19.186142    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:12:19.186518 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:19.186209    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 04:12:19.186518 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:19.186214    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs podName:ffa6e27a-0769-421d-a24d-3a9138bec37f nodeName:}" failed. No retries permitted until 2026-07-28 04:12:51.186192683 +0000 UTC m=+66.128032400 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs") pod "network-metrics-daemon-npqcf" (UID: "ffa6e27a-0769-421d-a24d-3a9138bec37f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 04:12:19.186518 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:19.186255    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret podName:95552c94-10a3-4446-8b07-fc7afaadc0d4 nodeName:}" failed. No retries permitted until 2026-07-28 04:12:51.186241537 +0000 UTC m=+66.128081236 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret") pod "global-pull-secret-syncer-wgdm4" (UID: "95552c94-10a3-4446-8b07-fc7afaadc0d4") : object "kube-system"/"original-pull-secret" not registered
Jul 28 04:12:19.387751 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.387715    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24wnf\" (UniqueName: \"kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf\") pod \"network-check-target-5rg2c\" (UID: \"c52368c0-8140-4957-b423-dcd1dead91b5\") " pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:19.388010 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:19.387907    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 04:12:19.388010 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:19.387930    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 04:12:19.388010 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:19.387944    2574 projected.go:196] Error preparing data for projected volume kube-api-access-24wnf for pod openshift-network-diagnostics/network-check-target-5rg2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:12:19.388010 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:19.388007    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf podName:c52368c0-8140-4957-b423-dcd1dead91b5 nodeName:}" failed. No retries permitted until 2026-07-28 04:12:51.387992953 +0000 UTC m=+66.329832656 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-24wnf" (UniqueName: "kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf") pod "network-check-target-5rg2c" (UID: "c52368c0-8140-4957-b423-dcd1dead91b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 04:12:19.417212 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.417182    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:19.505978 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.505957    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:19.610597 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.610567    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:19.617064 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.617047    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:19.617144 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.617137    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:19.617181 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:19.617150    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:12:19.617232 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:19.617214    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5rg2c" podUID="c52368c0-8140-4957-b423-dcd1dead91b5"
Jul 28 04:12:19.617267 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.617237    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:19.617338 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:19.617320    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgdm4" podUID="95552c94-10a3-4446-8b07-fc7afaadc0d4"
Jul 28 04:12:19.911862 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.911843    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-204.ec2.internal" event="NodeReady"
Jul 28 04:12:19.911981 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.911926    2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 04:12:19.950428 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.950406    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vqdlc"]
Jul 28 04:12:19.963069 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.963046    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hj7ns"]
Jul 28 04:12:19.963197 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.963178    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:19.965907 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.965184    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 04:12:19.965907 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.965498    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 04:12:19.965907 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.965692    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7zgg8\""
Jul 28 04:12:19.985509 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.985486    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vqdlc"]
Jul 28 04:12:19.985603 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.985531    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hj7ns"]
Jul 28 04:12:19.985657 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.985607    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hj7ns"
Jul 28 04:12:19.987992 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.987964    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 04:12:19.988326 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.988080    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 04:12:19.988326 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.988096    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 04:12:19.988326 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:19.988102    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-69gm8\""
Jul 28 04:12:20.091946 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.091908    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw96m\" (UniqueName: \"kubernetes.io/projected/a66b7189-fce5-4c9e-9722-188b5c351b16-kube-api-access-gw96m\") pod \"ingress-canary-hj7ns\" (UID: \"a66b7189-fce5-4c9e-9722-188b5c351b16\") " pod="openshift-ingress-canary/ingress-canary-hj7ns"
Jul 28 04:12:20.092071 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.091961    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:20.092071 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.092023    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a66b7189-fce5-4c9e-9722-188b5c351b16-cert\") pod \"ingress-canary-hj7ns\" (UID: \"a66b7189-fce5-4c9e-9722-188b5c351b16\") " pod="openshift-ingress-canary/ingress-canary-hj7ns"
Jul 28 04:12:20.092166 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.092102    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3411355c-8caa-429b-88cc-97ddaabc7760-config-volume\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:20.092166 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.092133    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj2tv\" (UniqueName: \"kubernetes.io/projected/3411355c-8caa-429b-88cc-97ddaabc7760-kube-api-access-gj2tv\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:20.092166 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.092155    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3411355c-8caa-429b-88cc-97ddaabc7760-tmp-dir\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:20.192654 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.192605    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw96m\" (UniqueName: \"kubernetes.io/projected/a66b7189-fce5-4c9e-9722-188b5c351b16-kube-api-access-gw96m\") pod \"ingress-canary-hj7ns\" (UID: \"a66b7189-fce5-4c9e-9722-188b5c351b16\") " pod="openshift-ingress-canary/ingress-canary-hj7ns"
Jul 28 04:12:20.192654 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.192642    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:20.193132 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.192676    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a66b7189-fce5-4c9e-9722-188b5c351b16-cert\") pod \"ingress-canary-hj7ns\" (UID: \"a66b7189-fce5-4c9e-9722-188b5c351b16\") " pod="openshift-ingress-canary/ingress-canary-hj7ns"
Jul 28 04:12:20.193132 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.192713    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3411355c-8caa-429b-88cc-97ddaabc7760-config-volume\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:20.193132 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:20.192778    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 04:12:20.193132 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.192838    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj2tv\" (UniqueName: \"kubernetes.io/projected/3411355c-8caa-429b-88cc-97ddaabc7760-kube-api-access-gj2tv\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:20.193132 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:20.192848    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls podName:3411355c-8caa-429b-88cc-97ddaabc7760 nodeName:}" failed. No retries permitted until 2026-07-28 04:12:20.692826564 +0000 UTC m=+35.634666265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls") pod "dns-default-vqdlc" (UID: "3411355c-8caa-429b-88cc-97ddaabc7760") : secret "dns-default-metrics-tls" not found
Jul 28 04:12:20.193132 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.192886    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3411355c-8caa-429b-88cc-97ddaabc7760-tmp-dir\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:20.193132 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.193112    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3411355c-8caa-429b-88cc-97ddaabc7760-tmp-dir\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:20.193348 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.193162    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3411355c-8caa-429b-88cc-97ddaabc7760-config-volume\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:20.196718 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.196596    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a66b7189-fce5-4c9e-9722-188b5c351b16-cert\") pod \"ingress-canary-hj7ns\" (UID: \"a66b7189-fce5-4c9e-9722-188b5c351b16\") " pod="openshift-ingress-canary/ingress-canary-hj7ns"
Jul 28 04:12:20.202339 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.202321    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw96m\" (UniqueName: \"kubernetes.io/projected/a66b7189-fce5-4c9e-9722-188b5c351b16-kube-api-access-gw96m\") pod \"ingress-canary-hj7ns\" (UID: \"a66b7189-fce5-4c9e-9722-188b5c351b16\") " pod="openshift-ingress-canary/ingress-canary-hj7ns"
Jul 28 04:12:20.207059 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.207041    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj2tv\" (UniqueName: \"kubernetes.io/projected/3411355c-8caa-429b-88cc-97ddaabc7760-kube-api-access-gj2tv\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:20.294208 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.294188    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hj7ns"
Jul 28 04:12:20.484975 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.484899    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hj7ns"]
Jul 28 04:12:20.488388 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:12:20.488351    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda66b7189_fce5_4c9e_9722_188b5c351b16.slice/crio-d3203d57f6856b543485d29e1bc1ad6e3ea8f70d4985708a92c404379a559117 WatchSource:0}: Error finding container d3203d57f6856b543485d29e1bc1ad6e3ea8f70d4985708a92c404379a559117: Status 404 returned error can't find the container with id d3203d57f6856b543485d29e1bc1ad6e3ea8f70d4985708a92c404379a559117
Jul 28 04:12:20.695634 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.695608    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:20.695772 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:20.695755    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 04:12:20.695814 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:20.695810    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls podName:3411355c-8caa-429b-88cc-97ddaabc7760 nodeName:}" failed. No retries permitted until 2026-07-28 04:12:21.695796221 +0000 UTC m=+36.637635922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls") pod "dns-default-vqdlc" (UID: "3411355c-8caa-429b-88cc-97ddaabc7760") : secret "dns-default-metrics-tls" not found
Jul 28 04:12:20.779703 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.779678    2574 generic.go:356] "Generic (PLEG): container finished" podID="679a6032-e93c-4030-b74b-6c5cdf46b470" containerID="a824d677ac0c7c4d855d6c91ee117c8ca76664bb653a3caeb6dc398b93131145" exitCode=0
Jul 28 04:12:20.779812 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.779723    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrsls" event={"ID":"679a6032-e93c-4030-b74b-6c5cdf46b470","Type":"ContainerDied","Data":"a824d677ac0c7c4d855d6c91ee117c8ca76664bb653a3caeb6dc398b93131145"}
Jul 28 04:12:20.780903 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:20.780877    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hj7ns" event={"ID":"a66b7189-fce5-4c9e-9722-188b5c351b16","Type":"ContainerStarted","Data":"d3203d57f6856b543485d29e1bc1ad6e3ea8f70d4985708a92c404379a559117"}
Jul 28 04:12:21.614947 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:21.614916    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:21.614947 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:21.614940    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:21.615482 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:21.615158    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:21.617018 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:21.616996    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 04:12:21.617018 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:21.617003    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 04:12:21.617276 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:21.617259    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 04:12:21.618311 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:21.618082    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9nvrd\""
Jul 28 04:12:21.618311 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:21.618215    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 04:12:21.618549 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:21.618313    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x5zc2\""
Jul 28 04:12:21.700258 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:21.700235    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:21.700430 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:21.700412    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 04:12:21.700496 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:21.700486    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls podName:3411355c-8caa-429b-88cc-97ddaabc7760 nodeName:}" failed. No retries permitted until 2026-07-28 04:12:23.700464857 +0000 UTC m=+38.642304558 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls") pod "dns-default-vqdlc" (UID: "3411355c-8caa-429b-88cc-97ddaabc7760") : secret "dns-default-metrics-tls" not found
Jul 28 04:12:21.786818 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:21.786791    2574 generic.go:356] "Generic (PLEG): container finished" podID="679a6032-e93c-4030-b74b-6c5cdf46b470" containerID="81eaabb82716a757157ecbbc71a3bf46d3ddb3300fc1cccf105fb70335a04855" exitCode=0
Jul 28 04:12:21.786933 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:21.786828    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrsls" event={"ID":"679a6032-e93c-4030-b74b-6c5cdf46b470","Type":"ContainerDied","Data":"81eaabb82716a757157ecbbc71a3bf46d3ddb3300fc1cccf105fb70335a04855"}
Jul 28 04:12:22.249912 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:22.249680    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:22.300184 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:22.300158    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:22.793314 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:22.793275    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mrsls" event={"ID":"679a6032-e93c-4030-b74b-6c5cdf46b470","Type":"ContainerStarted","Data":"32d2227be2181a80fbae39fbaa0c1366e1459112b13d0fbec3393f31997a9385"}
Jul 28 04:12:22.813771 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:22.813730    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mrsls" podStartSLOduration=9.525361913 podStartE2EDuration="37.813711981s" totalImagesPullingTime="28.288350068s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 04:11:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:12:22.813249065 +0000 UTC m=+37.755088785" watchObservedRunningTime="2026-07-28 04:12:22.813711981 +0000 UTC m=+37.755551704"
Jul 28 04:12:22.933731 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:22.933684    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:22.985442 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:22.985405    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:23.717120 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:23.717083    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:23.717255 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:23.717211    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 04:12:23.717296 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:23.717263    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls podName:3411355c-8caa-429b-88cc-97ddaabc7760 nodeName:}" failed. No retries permitted until 2026-07-28 04:12:27.717250365 +0000 UTC m=+42.659090066 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls") pod "dns-default-vqdlc" (UID: "3411355c-8caa-429b-88cc-97ddaabc7760") : secret "dns-default-metrics-tls" not found
Jul 28 04:12:23.796782 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:23.796757    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hj7ns" event={"ID":"a66b7189-fce5-4c9e-9722-188b5c351b16","Type":"ContainerStarted","Data":"dd4334db81061340e878f3ab8ed09395e3f1608a860e59f04809263a4c222fd8"}
Jul 28 04:12:23.810267 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:23.810229    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hj7ns" podStartSLOduration=2.112172528 podStartE2EDuration="4.810213897s" totalImagesPullingTime="2.698041369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:12:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:12:23.809124697 +0000 UTC m=+38.750964417" watchObservedRunningTime="2026-07-28 04:12:23.810213897 +0000 UTC m=+38.752053614"
Jul 28 04:12:25.269661 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:25.269626    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:25.315501 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:25.315475    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:26.186023 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:26.185990    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:26.234455 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:26.234429    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:27.743292 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:27.743260    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:27.743669 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:27.743402    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 04:12:27.743669 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:27.743448    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls podName:3411355c-8caa-429b-88cc-97ddaabc7760 nodeName:}" failed. No retries permitted until 2026-07-28 04:12:35.7434359 +0000 UTC m=+50.685275597 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls") pod "dns-default-vqdlc" (UID: "3411355c-8caa-429b-88cc-97ddaabc7760") : secret "dns-default-metrics-tls" not found
Jul 28 04:12:31.120665 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:31.120633    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:31.220701 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:31.220663    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:35.797373 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:35.797324    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:35.797760 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:35.797471    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 04:12:35.797760 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:35.797527    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls podName:3411355c-8caa-429b-88cc-97ddaabc7760 nodeName:}" failed. No retries permitted until 2026-07-28 04:12:51.797509278 +0000 UTC m=+66.739348992 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls") pod "dns-default-vqdlc" (UID: "3411355c-8caa-429b-88cc-97ddaabc7760") : secret "dns-default-metrics-tls" not found
Jul 28 04:12:41.523329 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:41.523291    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:41.622738 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:41.622703    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:44.773904 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:44.773878    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tb5v6"
Jul 28 04:12:44.921619 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:44.921591    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:44.980620 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:44.980595    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:48.213242 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:48.213208    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:48.268840 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:48.268797    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:48.316886 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:48.316866    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:51.204065 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.204036    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:12:51.204439 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.204081    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:51.206887 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.206871    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 04:12:51.206949 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.206933    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 04:12:51.214618 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:51.214600    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 04:12:51.214682 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:51.214667    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs podName:ffa6e27a-0769-421d-a24d-3a9138bec37f nodeName:}" failed. No retries permitted until 2026-07-28 04:13:55.214646253 +0000 UTC m=+130.156485964 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs") pod "network-metrics-daemon-npqcf" (UID: "ffa6e27a-0769-421d-a24d-3a9138bec37f") : secret "metrics-daemon-secret" not found
Jul 28 04:12:51.218064 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.218049    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95552c94-10a3-4446-8b07-fc7afaadc0d4-original-pull-secret\") pod \"global-pull-secret-syncer-wgdm4\" (UID: \"95552c94-10a3-4446-8b07-fc7afaadc0d4\") " pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:51.341994 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.341970    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgdm4"
Jul 28 04:12:51.406704 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.404789    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24wnf\" (UniqueName: \"kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf\") pod \"network-check-target-5rg2c\" (UID: \"c52368c0-8140-4957-b423-dcd1dead91b5\") " pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:51.407421 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.407199    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 04:12:51.421178 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.421155    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 04:12:51.432093 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.432047    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24wnf\" (UniqueName: \"kubernetes.io/projected/c52368c0-8140-4957-b423-dcd1dead91b5-kube-api-access-24wnf\") pod \"network-check-target-5rg2c\" (UID: \"c52368c0-8140-4957-b423-dcd1dead91b5\") " pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:51.474009 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.473954    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wgdm4"]
Jul 28 04:12:51.478049 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:12:51.478019    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95552c94_10a3_4446_8b07_fc7afaadc0d4.slice/crio-e54b62dd0bd2dae74d765aafe3255dad0641993757299ca8127aef994dec0a09 WatchSource:0}: Error finding container e54b62dd0bd2dae74d765aafe3255dad0641993757299ca8127aef994dec0a09: Status 404 returned error can't find the container with id e54b62dd0bd2dae74d765aafe3255dad0641993757299ca8127aef994dec0a09
Jul 28 04:12:51.634091 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.634072    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9nvrd\""
Jul 28 04:12:51.639818 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.639798    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:51.750673 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.750616    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5rg2c"]
Jul 28 04:12:51.754744 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:12:51.754722    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc52368c0_8140_4957_b423_dcd1dead91b5.slice/crio-7e112939d32b106f5f9e5ac64b0aa5074dc084fededa9f5eba840c8b78087caf WatchSource:0}: Error finding container 7e112939d32b106f5f9e5ac64b0aa5074dc084fededa9f5eba840c8b78087caf: Status 404 returned error can't find the container with id 7e112939d32b106f5f9e5ac64b0aa5074dc084fededa9f5eba840c8b78087caf
Jul 28 04:12:51.808168 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.808145    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:12:51.808277 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:51.808264    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 04:12:51.808324 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:12:51.808319    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls podName:3411355c-8caa-429b-88cc-97ddaabc7760 nodeName:}" failed. No retries permitted until 2026-07-28 04:13:23.80830477 +0000 UTC m=+98.750144468 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls") pod "dns-default-vqdlc" (UID: "3411355c-8caa-429b-88cc-97ddaabc7760") : secret "dns-default-metrics-tls" not found
Jul 28 04:12:51.856114 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.856087    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5rg2c" event={"ID":"c52368c0-8140-4957-b423-dcd1dead91b5","Type":"ContainerStarted","Data":"7e112939d32b106f5f9e5ac64b0aa5074dc084fededa9f5eba840c8b78087caf"}
Jul 28 04:12:51.857040 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:51.857020    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wgdm4" event={"ID":"95552c94-10a3-4446-8b07-fc7afaadc0d4","Type":"ContainerStarted","Data":"e54b62dd0bd2dae74d765aafe3255dad0641993757299ca8127aef994dec0a09"}
Jul 28 04:12:55.211429 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:55.211393    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:55.264427 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:55.264395    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:12:56.871600 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:56.871558    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5rg2c" event={"ID":"c52368c0-8140-4957-b423-dcd1dead91b5","Type":"ContainerStarted","Data":"7a222b46a46b8425d559a672397c6fd502fb3e592fa60a4af39ec497183c1a8f"}
Jul 28 04:12:56.872032 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:56.871633    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:12:56.872953 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:56.872930    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wgdm4" event={"ID":"95552c94-10a3-4446-8b07-fc7afaadc0d4","Type":"ContainerStarted","Data":"d0b944d641438dad47efd5f0f9c0604c6aa2b3030c4e54b93a34b5312a9d6c50"}
Jul 28 04:12:56.885490 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:56.885449    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5rg2c" podStartSLOduration=67.46113895 podStartE2EDuration="1m11.885434921s" totalImagesPullingTime="4.424295971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:12:56.884630361 +0000 UTC m=+71.826470082" watchObservedRunningTime="2026-07-28 04:12:56.885434921 +0000 UTC m=+71.827274640"
Jul 28 04:12:56.896814 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:12:56.896767    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wgdm4" podStartSLOduration=67.194813956 podStartE2EDuration="1m11.8967563s" totalImagesPullingTime="4.701942344s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 04:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:12:56.896655875 +0000 UTC m=+71.838495597" watchObservedRunningTime="2026-07-28 04:12:56.8967563 +0000 UTC m=+71.838596020"
Jul 28 04:13:18.642882 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:18.642769    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:13:18.690458 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:18.690428    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:13:23.903678 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:23.903650    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:13:23.904017 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:13:23.903760    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 04:13:23.904017 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:13:23.903809    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls podName:3411355c-8caa-429b-88cc-97ddaabc7760 nodeName:}" failed. No retries permitted until 2026-07-28 04:14:27.903794591 +0000 UTC m=+162.845634289 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls") pod "dns-default-vqdlc" (UID: "3411355c-8caa-429b-88cc-97ddaabc7760") : secret "dns-default-metrics-tls" not found
Jul 28 04:13:27.879723 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:27.879689    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5rg2c"
Jul 28 04:13:46.316480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.316449    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k"]
Jul 28 04:13:46.318080 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.318064    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k"
Jul 28 04:13:46.320368 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.320342    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 04:13:46.320801 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.320784    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 04:13:46.320978 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.320961    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"]
Jul 28 04:13:46.321525 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.321507    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-vvx88\""
Jul 28 04:13:46.321606 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.321530    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 04:13:46.321606 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.321553    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 28 04:13:46.322676 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.322663    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.324919 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.324769    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 28 04:13:46.324919 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.324888    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 28 04:13:46.325115 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.325100    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 28 04:13:46.325294 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.325276    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 28 04:13:46.329927 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.329910    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k"]
Jul 28 04:13:46.333867 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.333846    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"]
Jul 28 04:13:46.446717 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.446696    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8f78e5-c82f-4564-8f8b-43c779cf52db-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.446828 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.446724    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9vqf\" (UniqueName: \"kubernetes.io/projected/1a8f78e5-c82f-4564-8f8b-43c779cf52db-kube-api-access-j9vqf\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.446828 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.446756    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1a8f78e5-c82f-4564-8f8b-43c779cf52db-hub\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.446828 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.446778    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1a8f78e5-c82f-4564-8f8b-43c779cf52db-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.446996 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.446821    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1a8f78e5-c82f-4564-8f8b-43c779cf52db-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.446996 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.446859    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1a8f78e5-c82f-4564-8f8b-43c779cf52db-ca\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.446996 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.446900    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9e113b12-8c0b-4518-97e9-d6e7cea40fc7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k\" (UID: \"9e113b12-8c0b-4518-97e9-d6e7cea40fc7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k"
Jul 28 04:13:46.446996 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.446930    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xzz2\" (UniqueName: \"kubernetes.io/projected/9e113b12-8c0b-4518-97e9-d6e7cea40fc7-kube-api-access-2xzz2\") pod \"managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k\" (UID: \"9e113b12-8c0b-4518-97e9-d6e7cea40fc7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k"
Jul 28 04:13:46.547762 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.547731    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8f78e5-c82f-4564-8f8b-43c779cf52db-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.547762 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.547768    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9vqf\" (UniqueName: \"kubernetes.io/projected/1a8f78e5-c82f-4564-8f8b-43c779cf52db-kube-api-access-j9vqf\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.547870 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.547796    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1a8f78e5-c82f-4564-8f8b-43c779cf52db-hub\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.547870 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.547822    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1a8f78e5-c82f-4564-8f8b-43c779cf52db-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.547870 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.547849    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1a8f78e5-c82f-4564-8f8b-43c779cf52db-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.547996 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.547874    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1a8f78e5-c82f-4564-8f8b-43c779cf52db-ca\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.547996 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.547910    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9e113b12-8c0b-4518-97e9-d6e7cea40fc7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k\" (UID: \"9e113b12-8c0b-4518-97e9-d6e7cea40fc7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k"
Jul 28 04:13:46.547996 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.547935    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xzz2\" (UniqueName: \"kubernetes.io/projected/9e113b12-8c0b-4518-97e9-d6e7cea40fc7-kube-api-access-2xzz2\") pod \"managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k\" (UID: \"9e113b12-8c0b-4518-97e9-d6e7cea40fc7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k"
Jul 28 04:13:46.548505 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.548476    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8f78e5-c82f-4564-8f8b-43c779cf52db-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.550398 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.550378    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1a8f78e5-c82f-4564-8f8b-43c779cf52db-ca\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.550553 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.550533    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1a8f78e5-c82f-4564-8f8b-43c779cf52db-hub\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.550617 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.550599    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9e113b12-8c0b-4518-97e9-d6e7cea40fc7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k\" (UID: \"9e113b12-8c0b-4518-97e9-d6e7cea40fc7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k"
Jul 28 04:13:46.550686 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.550666    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1a8f78e5-c82f-4564-8f8b-43c779cf52db-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.550788 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.550774    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1a8f78e5-c82f-4564-8f8b-43c779cf52db-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.554239 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.554218    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xzz2\" (UniqueName: \"kubernetes.io/projected/9e113b12-8c0b-4518-97e9-d6e7cea40fc7-kube-api-access-2xzz2\") pod \"managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k\" (UID: \"9e113b12-8c0b-4518-97e9-d6e7cea40fc7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k"
Jul 28 04:13:46.554462 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.554437    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9vqf\" (UniqueName: \"kubernetes.io/projected/1a8f78e5-c82f-4564-8f8b-43c779cf52db-kube-api-access-j9vqf\") pod \"cluster-proxy-proxy-agent-7f596f584c-whrvw\" (UID: \"1a8f78e5-c82f-4564-8f8b-43c779cf52db\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.643547 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.643487    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k"
Jul 28 04:13:46.649004 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.648981    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:13:46.765136 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.765065    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"]
Jul 28 04:13:46.769906 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:13:46.769881    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a8f78e5_c82f_4564_8f8b_43c779cf52db.slice/crio-121e3ccf886c6f570a61048397a0400a5c8e75074e3e0483fb59189d1fb6d350 WatchSource:0}: Error finding container 121e3ccf886c6f570a61048397a0400a5c8e75074e3e0483fb59189d1fb6d350: Status 404 returned error can't find the container with id 121e3ccf886c6f570a61048397a0400a5c8e75074e3e0483fb59189d1fb6d350
Jul 28 04:13:46.779989 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.779967    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k"]
Jul 28 04:13:46.782595 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:13:46.782572    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e113b12_8c0b_4518_97e9_d6e7cea40fc7.slice/crio-1bd48bee249d847140bf70500ddbb9ecb4f0696e8fea1f884d3171510eb803a0 WatchSource:0}: Error finding container 1bd48bee249d847140bf70500ddbb9ecb4f0696e8fea1f884d3171510eb803a0: Status 404 returned error can't find the container with id 1bd48bee249d847140bf70500ddbb9ecb4f0696e8fea1f884d3171510eb803a0
Jul 28 04:13:46.982545 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.982493    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw" event={"ID":"1a8f78e5-c82f-4564-8f8b-43c779cf52db","Type":"ContainerStarted","Data":"121e3ccf886c6f570a61048397a0400a5c8e75074e3e0483fb59189d1fb6d350"}
Jul 28 04:13:46.983407 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:46.983385    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k" event={"ID":"9e113b12-8c0b-4518-97e9-d6e7cea40fc7","Type":"ContainerStarted","Data":"1bd48bee249d847140bf70500ddbb9ecb4f0696e8fea1f884d3171510eb803a0"}
Jul 28 04:13:50.997179 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:50.997106    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw" event={"ID":"1a8f78e5-c82f-4564-8f8b-43c779cf52db","Type":"ContainerStarted","Data":"9b34dcc44c0eae9207a845cbb7a72e8cf71104da4f573732f59492b8e4fe50a1"}
Jul 28 04:13:50.997179 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:50.997158    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw" event={"ID":"1a8f78e5-c82f-4564-8f8b-43c779cf52db","Type":"ContainerStarted","Data":"77bfde180b2240b4664428551e29765e13596439ed185f7c05ee85cbbcfa0e8e"}
Jul 28 04:13:50.997179 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:50.997178    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw" event={"ID":"1a8f78e5-c82f-4564-8f8b-43c779cf52db","Type":"ContainerStarted","Data":"2c1a55d5b913155da49e9ac1b35163b8d3d8c829ae55d69828beddf98e013f18"}
Jul 28 04:13:50.998464 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:50.998441    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k" event={"ID":"9e113b12-8c0b-4518-97e9-d6e7cea40fc7","Type":"ContainerStarted","Data":"3d1e8fc6a55d52d642c7d4fca9d45735261716ec03a1393f2213cda7d5058035"}
Jul 28 04:13:51.013287 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:51.013252    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw" podStartSLOduration=1.13841263 podStartE2EDuration="5.013239158s" totalImagesPullingTime="3.874826528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:13:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:13:51.012099672 +0000 UTC m=+125.953939393" watchObservedRunningTime="2026-07-28 04:13:51.013239158 +0000 UTC m=+125.955078878"
Jul 28 04:13:51.025486 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:51.025449    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k" podStartSLOduration=1.173267653 podStartE2EDuration="5.025437563s" totalImagesPullingTime="3.85216991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:13:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:13:51.025207881 +0000 UTC m=+125.967047602" watchObservedRunningTime="2026-07-28 04:13:51.025437563 +0000 UTC m=+125.967277283"
Jul 28 04:13:55.306753 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:13:55.306717    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:13:55.307196 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:13:55.306861    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 04:13:55.307196 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:13:55.306924    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs podName:ffa6e27a-0769-421d-a24d-3a9138bec37f nodeName:}" failed. No retries permitted until 2026-07-28 04:15:57.306909171 +0000 UTC m=+252.248748874 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs") pod "network-metrics-daemon-npqcf" (UID: "ffa6e27a-0769-421d-a24d-3a9138bec37f") : secret "metrics-daemon-secret" not found
Jul 28 04:14:22.978132 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:14:22.978096    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vqdlc" podUID="3411355c-8caa-429b-88cc-97ddaabc7760"
Jul 28 04:14:23.083880 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:14:23.083845    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:14:24.635859 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:14:24.635826    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-npqcf" podUID="ffa6e27a-0769-421d-a24d-3a9138bec37f"
Jul 28 04:14:27.923447 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:14:27.923407    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:14:27.923808 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:14:27.923522    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 04:14:27.923808 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:14:27.923581    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls podName:3411355c-8caa-429b-88cc-97ddaabc7760 nodeName:}" failed. No retries permitted until 2026-07-28 04:16:29.923566286 +0000 UTC m=+284.865405985 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls") pod "dns-default-vqdlc" (UID: "3411355c-8caa-429b-88cc-97ddaabc7760") : secret "dns-default-metrics-tls" not found
Jul 28 04:14:36.614277 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:14:36.614247    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:15:37.168269 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:37.168242    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:37.247839 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:37.247807    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:39.683642 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:39.683613    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:39.733018 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:39.732995    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:42.515183 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.515142    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:42.569301 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.569274    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-76frw"]
Jul 28 04:15:42.571270 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.571255    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.573024 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.573002    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-data-volume\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.573188 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.573030    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.573188 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.573066    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.573188 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.573090    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r227x\" (UniqueName: \"kubernetes.io/projected/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-api-access-r227x\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.573389 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.573183    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-crio-socket\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.573389 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.573246    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 04:15:42.573389 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.573259    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 04:15:42.573389 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.573278    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 04:15:42.573555 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.573541    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tkv7l\""
Jul 28 04:15:42.573896 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.573881    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 04:15:42.581202 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.581180    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-76frw"]
Jul 28 04:15:42.589510 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.589490    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:42.640985 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.640959    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:42.674126 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.674105    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.674202 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.674132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r227x\" (UniqueName: \"kubernetes.io/projected/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-api-access-r227x\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.674202 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.674168    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-crio-socket\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.674202 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.674186    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-data-volume\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.674355 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.674241    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-crio-socket\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.674355 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:42.674250    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 04:15:42.674355 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.674274    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.674355 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:42.674317    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls podName:c5ff3174-cacb-4bc7-bb73-c6e89b47d427 nodeName:}" failed. No retries permitted until 2026-07-28 04:15:43.174299985 +0000 UTC m=+238.116139698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls") pod "insights-runtime-extractor-76frw" (UID: "c5ff3174-cacb-4bc7-bb73-c6e89b47d427") : secret "insights-runtime-extractor-tls" not found
Jul 28 04:15:42.674355 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:42.674333    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 04:15:42.674619 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:42.674412    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm podName:c5ff3174-cacb-4bc7-bb73-c6e89b47d427 nodeName:}" failed. No retries permitted until 2026-07-28 04:15:43.174398865 +0000 UTC m=+238.116238777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm") pod "insights-runtime-extractor-76frw" (UID: "c5ff3174-cacb-4bc7-bb73-c6e89b47d427") : configmap "kube-rbac-proxy" not found
Jul 28 04:15:42.674619 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.674452    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-data-volume\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.681489 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.681474    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r227x\" (UniqueName: \"kubernetes.io/projected/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-api-access-r227x\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:42.884042 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.884016    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:42.958979 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:42.958956    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:43.112536 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:43.112512    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:43.177740 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:43.177689    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:43.177740 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:43.177734    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:43.177842 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:43.177800    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 04:15:43.177842 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:43.177822    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 04:15:43.177906 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:43.177847    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm podName:c5ff3174-cacb-4bc7-bb73-c6e89b47d427 nodeName:}" failed. No retries permitted until 2026-07-28 04:15:44.177833922 +0000 UTC m=+239.119673623 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm") pod "insights-runtime-extractor-76frw" (UID: "c5ff3174-cacb-4bc7-bb73-c6e89b47d427") : configmap "kube-rbac-proxy" not found
Jul 28 04:15:43.177906 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:43.177873    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls podName:c5ff3174-cacb-4bc7-bb73-c6e89b47d427 nodeName:}" failed. No retries permitted until 2026-07-28 04:15:44.177859709 +0000 UTC m=+239.119699411 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls") pod "insights-runtime-extractor-76frw" (UID: "c5ff3174-cacb-4bc7-bb73-c6e89b47d427") : secret "insights-runtime-extractor-tls" not found
Jul 28 04:15:43.213780 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:43.213760    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:43.663957 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:43.663927    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:43.763230 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:43.763199    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:44.184897 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:44.184863    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:44.185046 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:44.184917    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:44.185046 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:44.184976    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 04:15:44.185046 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:44.185001    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 04:15:44.185046 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:44.185043    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm podName:c5ff3174-cacb-4bc7-bb73-c6e89b47d427 nodeName:}" failed. No retries permitted until 2026-07-28 04:15:46.185027244 +0000 UTC m=+241.126866942 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm") pod "insights-runtime-extractor-76frw" (UID: "c5ff3174-cacb-4bc7-bb73-c6e89b47d427") : configmap "kube-rbac-proxy" not found
Jul 28 04:15:44.185200 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:44.185058    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls podName:c5ff3174-cacb-4bc7-bb73-c6e89b47d427 nodeName:}" failed. No retries permitted until 2026-07-28 04:15:46.185052248 +0000 UTC m=+241.126891946 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls") pod "insights-runtime-extractor-76frw" (UID: "c5ff3174-cacb-4bc7-bb73-c6e89b47d427") : secret "insights-runtime-extractor-tls" not found
Jul 28 04:15:44.699915 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:44.699873    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:44.799723 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:44.799691    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:45.291093 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:45.291066    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:45.391748 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:45.391710    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:45.559630 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:45.559569    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xgf87_3ec166b8-6374-4e81-9a43-65a6fe177e7a/dns-node-resolver/0.log"
Jul 28 04:15:46.053347 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:46.053300    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:46.160273 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:46.160248    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-27l4c_d2385840-55b1-4920-aa89-ec64f34697f6/node-ca/0.log"
Jul 28 04:15:46.199261 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:46.199241    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:46.199327 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:46.199282    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:46.199384 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:46.199348    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 04:15:46.199384 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:46.199377    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 04:15:46.199452 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:46.199419    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm podName:c5ff3174-cacb-4bc7-bb73-c6e89b47d427 nodeName:}" failed. No retries permitted until 2026-07-28 04:15:50.199405804 +0000 UTC m=+245.141245505 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm") pod "insights-runtime-extractor-76frw" (UID: "c5ff3174-cacb-4bc7-bb73-c6e89b47d427") : configmap "kube-rbac-proxy" not found
Jul 28 04:15:46.199452 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:46.199432    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls podName:c5ff3174-cacb-4bc7-bb73-c6e89b47d427 nodeName:}" failed. No retries permitted until 2026-07-28 04:15:50.199426505 +0000 UTC m=+245.141266202 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls") pod "insights-runtime-extractor-76frw" (UID: "c5ff3174-cacb-4bc7-bb73-c6e89b47d427") : secret "insights-runtime-extractor-tls" not found
Jul 28 04:15:46.839332 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:46.839267    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:46.979878 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:46.979832    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:47.089048 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:47.089021    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:47.159959 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:47.159887    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hj7ns_a66b7189-fce5-4c9e-9722-188b5c351b16/serve-healthcheck-canary/0.log"
Jul 28 04:15:47.189965 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:47.189939    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:47.242210 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:47.242187    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:47.315772 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:47.315750    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:47.539382 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:47.539331    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:48.744374 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:48.744319    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:48.944535 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:48.944501    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:49.008139 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:49.008048    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:49.146031 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:49.145990    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:49.347832 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:49.347803    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:49.447822 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:49.447791    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:49.548223 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:49.548195    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:50.226677 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:50.226644    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:50.227049 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:50.226701    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:50.227049 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:50.226760    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 04:15:50.227049 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:50.226821    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm podName:c5ff3174-cacb-4bc7-bb73-c6e89b47d427 nodeName:}" failed. No retries permitted until 2026-07-28 04:15:58.226806717 +0000 UTC m=+253.168646415 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm") pod "insights-runtime-extractor-76frw" (UID: "c5ff3174-cacb-4bc7-bb73-c6e89b47d427") : configmap "kube-rbac-proxy" not found
Jul 28 04:15:50.227049 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:50.226823    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 04:15:50.227049 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:15:50.226875    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls podName:c5ff3174-cacb-4bc7-bb73-c6e89b47d427 nodeName:}" failed. No retries permitted until 2026-07-28 04:15:58.226860639 +0000 UTC m=+253.168700347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls") pod "insights-runtime-extractor-76frw" (UID: "c5ff3174-cacb-4bc7-bb73-c6e89b47d427") : secret "insights-runtime-extractor-tls" not found
Jul 28 04:15:50.738450 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:50.738408    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:50.791520 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:50.791496    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:50.937868 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:50.937839    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:51.139243 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:51.139215    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:51.319600 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:51.319568    2574 generic.go:356] "Generic (PLEG): container finished" podID="9e113b12-8c0b-4518-97e9-d6e7cea40fc7" containerID="3d1e8fc6a55d52d642c7d4fca9d45735261716ec03a1393f2213cda7d5058035" exitCode=255
Jul 28 04:15:51.319960 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:51.319620    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k" event={"ID":"9e113b12-8c0b-4518-97e9-d6e7cea40fc7","Type":"ContainerDied","Data":"3d1e8fc6a55d52d642c7d4fca9d45735261716ec03a1393f2213cda7d5058035"}
Jul 28 04:15:51.319960 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:51.319892    2574 scope.go:117] "RemoveContainer" containerID="3d1e8fc6a55d52d642c7d4fca9d45735261716ec03a1393f2213cda7d5058035"
Jul 28 04:15:51.340425 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:51.340403    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:51.541417 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:51.541382    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:51.930532 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:51.930453    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:52.272894 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:52.272814    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:52.324087 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:52.324051    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-79fc9fc6f6-h7r2k" event={"ID":"9e113b12-8c0b-4518-97e9-d6e7cea40fc7","Type":"ContainerStarted","Data":"cfabe0ac9ffa68960c8ce79c2eb874aeb9f16f1c0311f762dcddcd0b8bb5bec0"}
Jul 28 04:15:52.751259 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:52.751227    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:52.942614 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:52.942590    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:53.043649 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:53.043629    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:53.143717 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:53.143692    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:53.340883 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:53.340821    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:53.540134 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:53.540104    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:54.542327 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:54.542298    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:54.938897 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:54.938818    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:55.136071 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:55.136041    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:55.361141 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:55.361118    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:55.897942 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:55.897912    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:55.998631 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:55.998605    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:56.534854 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:56.534824    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:56.738420 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:56.738395    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:57.134818 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:57.134794    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:57.335804 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:57.335776    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:57.372566 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:57.372543    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:15:57.374699 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:57.374678    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa6e27a-0769-421d-a24d-3a9138bec37f-metrics-certs\") pod \"network-metrics-daemon-npqcf\" (UID: \"ffa6e27a-0769-421d-a24d-3a9138bec37f\") " pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:15:57.539999 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:57.539973    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:15:57.616446 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:57.616424    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x5zc2\""
Jul 28 04:15:57.624843 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:57.624823    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-npqcf"
Jul 28 04:15:57.739310 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:57.739283    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-npqcf"]
Jul 28 04:15:57.742568 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:15:57.742538    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa6e27a_0769_421d_a24d_3a9138bec37f.slice/crio-6bd3e0be89c14243fef18c75be1da0941d368487330d6a51b57c90ae6b25e537 WatchSource:0}: Error finding container 6bd3e0be89c14243fef18c75be1da0941d368487330d6a51b57c90ae6b25e537: Status 404 returned error can't find the container with id 6bd3e0be89c14243fef18c75be1da0941d368487330d6a51b57c90ae6b25e537
Jul 28 04:15:58.279792 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:58.279746    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:58.280232 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:58.279795    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:58.280373 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:58.280340    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:58.282305 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:58.282285    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ff3174-cacb-4bc7-bb73-c6e89b47d427-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-76frw\" (UID: \"c5ff3174-cacb-4bc7-bb73-c6e89b47d427\") " pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:58.341226 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:58.341196    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-npqcf" event={"ID":"ffa6e27a-0769-421d-a24d-3a9138bec37f","Type":"ContainerStarted","Data":"6bd3e0be89c14243fef18c75be1da0941d368487330d6a51b57c90ae6b25e537"}
Jul 28 04:15:58.481744 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:58.481719    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tkv7l\""
Jul 28 04:15:58.491221 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:58.491195    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-76frw"
Jul 28 04:15:58.777240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:58.777217    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-76frw"]
Jul 28 04:15:58.779938 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:15:58.779915    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ff3174_cacb_4bc7_bb73_c6e89b47d427.slice/crio-40076b564ae6bc4ecf4e625ee3604a5d9c7116a67169ef7d3f7af51426f4dfc8 WatchSource:0}: Error finding container 40076b564ae6bc4ecf4e625ee3604a5d9c7116a67169ef7d3f7af51426f4dfc8: Status 404 returned error can't find the container with id 40076b564ae6bc4ecf4e625ee3604a5d9c7116a67169ef7d3f7af51426f4dfc8
Jul 28 04:15:59.347841 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:59.347800    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-npqcf" event={"ID":"ffa6e27a-0769-421d-a24d-3a9138bec37f","Type":"ContainerStarted","Data":"8c66f7b77c04cdd0e38f0f826be69dacba68491b3acda593573ef0a01855fcb0"}
Jul 28 04:15:59.348428 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:59.347902    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-npqcf" event={"ID":"ffa6e27a-0769-421d-a24d-3a9138bec37f","Type":"ContainerStarted","Data":"f79a0d405b5ad56741cce68dde292c8be40cb7977e8c80a682208b4bab1c8579"}
Jul 28 04:15:59.349596 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:59.349569    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-76frw" event={"ID":"c5ff3174-cacb-4bc7-bb73-c6e89b47d427","Type":"ContainerStarted","Data":"40076b564ae6bc4ecf4e625ee3604a5d9c7116a67169ef7d3f7af51426f4dfc8"}
Jul 28 04:15:59.363145 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:15:59.362301    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-npqcf" podStartSLOduration=253.416707952 podStartE2EDuration="4m14.362280758s" totalImagesPullingTime="945.572806ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:15:59.360673176 +0000 UTC m=+254.302512893" watchObservedRunningTime="2026-07-28 04:15:59.362280758 +0000 UTC m=+254.304120480"
Jul 28 04:16:01.124928 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:01.124887    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:01.225024 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:01.224991    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:04.368970 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:04.368923    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-76frw" event={"ID":"c5ff3174-cacb-4bc7-bb73-c6e89b47d427","Type":"ContainerStarted","Data":"246be437eed134cba6e7f735a14d48a164a65bfeca82b8152364e676cbfe5ddd"}
Jul 28 04:16:06.547776 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:06.547743    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:06.644815 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:06.644778    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:07.379861 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:07.379776    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-76frw" event={"ID":"c5ff3174-cacb-4bc7-bb73-c6e89b47d427","Type":"ContainerStarted","Data":"65ac780dfc1f852da9cfd373e50f53d15caf2ed0b814182bdeb83ce017c4dd14"}
Jul 28 04:16:11.395132 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:11.395098    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-76frw" event={"ID":"c5ff3174-cacb-4bc7-bb73-c6e89b47d427","Type":"ContainerStarted","Data":"2c718ad88bc2d5f5798ab4bacf37fff5612aa1bad4d6c0da79937e0e6c911975"}
Jul 28 04:16:11.410930 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:11.410878    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-76frw" podStartSLOduration=17.771149256 podStartE2EDuration="29.410863119s" totalImagesPullingTime="11.639713863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 04:15:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:16:11.409865445 +0000 UTC m=+266.351705164" watchObservedRunningTime="2026-07-28 04:16:11.410863119 +0000 UTC m=+266.352702839"
Jul 28 04:16:12.842092 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:12.842061    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:12.891280 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:12.891242    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:13.017905 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:13.017880    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:15.837020 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:15.836989    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:15.939152 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:15.939124    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:16.038449 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:16.038422    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:18.790860 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:18.790828    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:20.980182 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:20.980149    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:21.189883 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.189851    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:21.206847 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.206826    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"]
Jul 28 04:16:21.209764 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.209749    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:21.211822 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.211802    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 04:16:21.212274 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.212253    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 04:16:21.212274 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.212271    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 04:16:21.212448 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.212337    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 04:16:21.212668 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.212649    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zqfcl\""
Jul 28 04:16:21.212797 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.212668    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 04:16:21.219225 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.219205    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"]
Jul 28 04:16:21.241455 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.241403    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d5m5q"]
Jul 28 04:16:21.244407 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.244390    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.246762 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.246745    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 04:16:21.246762 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.246757    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 04:16:21.247030 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.247014    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 04:16:21.247109 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.247042    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6692x\""
Jul 28 04:16:21.269763 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.269740    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:21.332118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.332097    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfrlt\" (UniqueName: \"kubernetes.io/projected/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-kube-api-access-rfrlt\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.332205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.332130    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz58p\" (UniqueName: \"kubernetes.io/projected/478a0703-2277-4856-aead-a0b3e7a01fc9-kube-api-access-sz58p\") pod \"openshift-state-metrics-b8f49f5c6-pkpjb\" (UID: \"478a0703-2277-4856-aead-a0b3e7a01fc9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:21.332205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.332148    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/478a0703-2277-4856-aead-a0b3e7a01fc9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-pkpjb\" (UID: \"478a0703-2277-4856-aead-a0b3e7a01fc9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:21.332205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.332166    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/478a0703-2277-4856-aead-a0b3e7a01fc9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-pkpjb\" (UID: \"478a0703-2277-4856-aead-a0b3e7a01fc9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:21.332205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.332183    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-root\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.332205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.332205    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.332401 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.332228    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-textfile\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.332401 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.332243    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-sys\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.332401 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.332263    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-tls\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.332401 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.332309    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-metrics-client-ca\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.332401 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.332328    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-wtmp\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.332401 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.332343    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/478a0703-2277-4856-aead-a0b3e7a01fc9-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-pkpjb\" (UID: \"478a0703-2277-4856-aead-a0b3e7a01fc9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:21.332401 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.332376    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-accelerators-collector-config\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.433089 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433067    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-metrics-client-ca\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.433197 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433096    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-wtmp\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.433197 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433115    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/478a0703-2277-4856-aead-a0b3e7a01fc9-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-pkpjb\" (UID: \"478a0703-2277-4856-aead-a0b3e7a01fc9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:21.433197 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433131    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-accelerators-collector-config\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.433197 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433165    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfrlt\" (UniqueName: \"kubernetes.io/projected/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-kube-api-access-rfrlt\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.433421 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433271    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-wtmp\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.433421 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433343    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz58p\" (UniqueName: \"kubernetes.io/projected/478a0703-2277-4856-aead-a0b3e7a01fc9-kube-api-access-sz58p\") pod \"openshift-state-metrics-b8f49f5c6-pkpjb\" (UID: \"478a0703-2277-4856-aead-a0b3e7a01fc9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:21.433421 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433397    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/478a0703-2277-4856-aead-a0b3e7a01fc9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-pkpjb\" (UID: \"478a0703-2277-4856-aead-a0b3e7a01fc9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:21.433576 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433432    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/478a0703-2277-4856-aead-a0b3e7a01fc9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-pkpjb\" (UID: \"478a0703-2277-4856-aead-a0b3e7a01fc9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:21.433576 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433457    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-root\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.433576 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433484    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.433576 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433522    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-textfile\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.433576 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-sys\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.433576 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433573    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-tls\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.433862 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433779    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-metrics-client-ca\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.433862 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433823    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-accelerators-collector-config\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.433971 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433892    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/478a0703-2277-4856-aead-a0b3e7a01fc9-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-pkpjb\" (UID: \"478a0703-2277-4856-aead-a0b3e7a01fc9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:21.434028 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.433969    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-sys\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.434081 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:16:21.434048    2574 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 28 04:16:21.434081 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.434071    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-textfile\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.434183 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:16:21.434131    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478a0703-2277-4856-aead-a0b3e7a01fc9-openshift-state-metrics-tls podName:478a0703-2277-4856-aead-a0b3e7a01fc9 nodeName:}" failed. No retries permitted until 2026-07-28 04:16:21.934106786 +0000 UTC m=+276.875946499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/478a0703-2277-4856-aead-a0b3e7a01fc9-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-pkpjb" (UID: "478a0703-2277-4856-aead-a0b3e7a01fc9") : secret "openshift-state-metrics-tls" not found
Jul 28 04:16:21.434183 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.434137    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-root\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.436034 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.436015    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.436119 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.436045    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/478a0703-2277-4856-aead-a0b3e7a01fc9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-pkpjb\" (UID: \"478a0703-2277-4856-aead-a0b3e7a01fc9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:21.436119 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.436098    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-node-exporter-tls\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.446621 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.446604    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfrlt\" (UniqueName: \"kubernetes.io/projected/2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc-kube-api-access-rfrlt\") pod \"node-exporter-d5m5q\" (UID: \"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc\") " pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.447056 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.447035    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz58p\" (UniqueName: \"kubernetes.io/projected/478a0703-2277-4856-aead-a0b3e7a01fc9-kube-api-access-sz58p\") pod \"openshift-state-metrics-b8f49f5c6-pkpjb\" (UID: \"478a0703-2277-4856-aead-a0b3e7a01fc9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:21.454920 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.454901    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:21.551742 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.551720    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:21.552640 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.552625    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d5m5q"
Jul 28 04:16:21.563255 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:16:21.563235    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a67d072_1cde_43ed_bb01_dfa4ff6f0dcc.slice/crio-08bfafa3833a15cb25faee9bff0d38dfd47eac409c5541d33eec04c1d13a706f WatchSource:0}: Error finding container 08bfafa3833a15cb25faee9bff0d38dfd47eac409c5541d33eec04c1d13a706f: Status 404 returned error can't find the container with id 08bfafa3833a15cb25faee9bff0d38dfd47eac409c5541d33eec04c1d13a706f
Jul 28 04:16:21.673483 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.673462    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:21.774177 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.774158    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:21.826256 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.826213    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:21.937317 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.937297    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/478a0703-2277-4856-aead-a0b3e7a01fc9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-pkpjb\" (UID: \"478a0703-2277-4856-aead-a0b3e7a01fc9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:21.939478 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:21.939453    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/478a0703-2277-4856-aead-a0b3e7a01fc9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-pkpjb\" (UID: \"478a0703-2277-4856-aead-a0b3e7a01fc9\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:22.118438 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.118344    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"
Jul 28 04:16:22.196617 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.196421    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:22.238307 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.238185    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb"]
Jul 28 04:16:22.240499 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:16:22.240479    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478a0703_2277_4856_aead_a0b3e7a01fc9.slice/crio-3f46fc0787e4c6a68ede5c4b8b0c62a2720ce01cdbb77ce3212826c8695a289b WatchSource:0}: Error finding container 3f46fc0787e4c6a68ede5c4b8b0c62a2720ce01cdbb77ce3212826c8695a289b: Status 404 returned error can't find the container with id 3f46fc0787e4c6a68ede5c4b8b0c62a2720ce01cdbb77ce3212826c8695a289b
Jul 28 04:16:22.284473 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.284450    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 04:16:22.287782 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.287762    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.290056 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.289859    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 04:16:22.290171 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.290136    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 04:16:22.290264 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.290227    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 04:16:22.290472 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.290448    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 04:16:22.290472 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.290477    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 04:16:22.290703 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.290688    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 04:16:22.290778 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.290708    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 04:16:22.290922 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.290900    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 04:16:22.290987 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.290925    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nx68t\""
Jul 28 04:16:22.291269 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.291130    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 04:16:22.301982 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.301922    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 04:16:22.340147 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.340117    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.340256 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.340157    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.340256 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.340201    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.340256 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.340227    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zq5f\" (UniqueName: \"kubernetes.io/projected/997996e2-1121-4647-8f6d-52dfd2ed897d-kube-api-access-9zq5f\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.340393 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.340253    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.340393 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.340322    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/997996e2-1121-4647-8f6d-52dfd2ed897d-config-out\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.340393 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.340355    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.340511 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.340402    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/997996e2-1121-4647-8f6d-52dfd2ed897d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.340511 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.340423    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.340511 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.340448    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.340511 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.340475    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-config-volume\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.340683 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.340509    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.340683 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.340568    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-web-config\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.430007 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.429936    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb" event={"ID":"478a0703-2277-4856-aead-a0b3e7a01fc9","Type":"ContainerStarted","Data":"55f1fc54ca568f10ac06b7d0f58afba3888708979d22810f8db72d895485ee39"}
Jul 28 04:16:22.430007 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.429991    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb" event={"ID":"478a0703-2277-4856-aead-a0b3e7a01fc9","Type":"ContainerStarted","Data":"f481c6fbf0c38a3aa5e7c075a8466440ad3b50e537019101d9df1a65508f8d3b"}
Jul 28 04:16:22.430188 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.430016    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb" event={"ID":"478a0703-2277-4856-aead-a0b3e7a01fc9","Type":"ContainerStarted","Data":"3f46fc0787e4c6a68ede5c4b8b0c62a2720ce01cdbb77ce3212826c8695a289b"}
Jul 28 04:16:22.431249 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.431221    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d5m5q" event={"ID":"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc","Type":"ContainerStarted","Data":"08bfafa3833a15cb25faee9bff0d38dfd47eac409c5541d33eec04c1d13a706f"}
Jul 28 04:16:22.441605 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.441579    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.441725 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.441623    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/997996e2-1121-4647-8f6d-52dfd2ed897d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.441791 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.441734    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.441791 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.441771    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.441909 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.441800    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-config-volume\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.441963 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.441909    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.442017 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.441980    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-web-config\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.442069 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.442015    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.442069 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.442051    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.442178 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.442105    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.442178 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.442114    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.442178 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.442133    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zq5f\" (UniqueName: \"kubernetes.io/projected/997996e2-1121-4647-8f6d-52dfd2ed897d-kube-api-access-9zq5f\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.442178 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.442161    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.442405 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.442214    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/997996e2-1121-4647-8f6d-52dfd2ed897d-config-out\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.442405 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:16:22.442260    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-trusted-ca-bundle podName:997996e2-1121-4647-8f6d-52dfd2ed897d nodeName:}" failed. No retries permitted until 2026-07-28 04:16:22.942240669 +0000 UTC m=+277.884080366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d") : configmap references non-existent config key: ca-bundle.crt
Jul 28 04:16:22.442529 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.442480    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.445353 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.445311    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.445517 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.445471    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.445939 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.445733    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/997996e2-1121-4647-8f6d-52dfd2ed897d-config-out\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.445939 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.445902    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-web-config\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.445939 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.445901    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/997996e2-1121-4647-8f6d-52dfd2ed897d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.447134 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.447087    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.447289 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.447271    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-config-volume\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.447742 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.447720    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.447844 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.447825    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.451707 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.451679    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zq5f\" (UniqueName: \"kubernetes.io/projected/997996e2-1121-4647-8f6d-52dfd2ed897d-kube-api-access-9zq5f\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.526184 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.526144    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:22.581003 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.580969    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:22.946491 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.946439    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:22.947185 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:22.947164    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:23.150496 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:23.150463    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:23.199710 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:23.199682    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:16:23.250899 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:23.250868    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:23.351224 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:23.351198    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:23.436950 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:23.436924    2574 generic.go:356] "Generic (PLEG): container finished" podID="2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc" containerID="a8cebd403255d53dfeb2dd0cf34be565a36402688ad74985c4e74e50c867aaba" exitCode=0
Jul 28 04:16:23.437042 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:23.436972    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d5m5q" event={"ID":"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc","Type":"ContainerDied","Data":"a8cebd403255d53dfeb2dd0cf34be565a36402688ad74985c4e74e50c867aaba"}
Jul 28 04:16:23.454496 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:23.454468    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:23.486080 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:23.486056    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 04:16:23.488418 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:16:23.488391    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod997996e2_1121_4647_8f6d_52dfd2ed897d.slice/crio-d37f1a114887a1fe6ab9e9918ad5a932087a3f1c9dbc2a89a0c8e74e33e5e0f9 WatchSource:0}: Error finding container d37f1a114887a1fe6ab9e9918ad5a932087a3f1c9dbc2a89a0c8e74e33e5e0f9: Status 404 returned error can't find the container with id d37f1a114887a1fe6ab9e9918ad5a932087a3f1c9dbc2a89a0c8e74e33e5e0f9
Jul 28 04:16:23.605078 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:23.605058    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:23.656906 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:23.656849    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:23.808319 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:23.808293    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:23.907545 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:23.907498    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:24.032263 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.032243    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:24.135744 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.135717    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:24.201380 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.201309    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-654777ddcb-bxmmw"]
Jul 28 04:16:24.204575 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.204560    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.206843 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.206826    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-66ndnjonlm4ro\""
Jul 28 04:16:24.206973 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.206954    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 04:16:24.207026 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.206975    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vrf6w\""
Jul 28 04:16:24.207085 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.207029    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 04:16:24.207857 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.207838    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 04:16:24.207931 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.207880    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 04:16:24.207987 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.207882    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 04:16:24.215146 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.215128    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-654777ddcb-bxmmw"]
Jul 28 04:16:24.221074 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.221056    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:24.257687 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.257665    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c12900c-a219-4f38-9b10-774b733f73b8-metrics-client-ca\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.257783 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.257707    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.257783 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.257735    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.257783 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.257771    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-tls\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.257931 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.257818    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.257931 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.257871    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.257931 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.257894    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n66hq\" (UniqueName: \"kubernetes.io/projected/6c12900c-a219-4f38-9b10-774b733f73b8-kube-api-access-n66hq\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.257931 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.257919    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-grpc-tls\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.277009 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.276985    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:24.330146 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.330124    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:24.358304 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.358282    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.358405 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.358313    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.358405 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.358337    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-tls\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.358545 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.358531    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.358593 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.358559    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.358593 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.358579    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n66hq\" (UniqueName: \"kubernetes.io/projected/6c12900c-a219-4f38-9b10-774b733f73b8-kube-api-access-n66hq\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.358751 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.358731    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-grpc-tls\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.358827 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.358807    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c12900c-a219-4f38-9b10-774b733f73b8-metrics-client-ca\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.359604 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.359544    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c12900c-a219-4f38-9b10-774b733f73b8-metrics-client-ca\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.361222 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.361199    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.361724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.361683    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-tls\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.362027 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.362007    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-grpc-tls\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.362206 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.362186    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.362518 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.362497    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.362788 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.362769    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6c12900c-a219-4f38-9b10-774b733f73b8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.365948 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.365928    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n66hq\" (UniqueName: \"kubernetes.io/projected/6c12900c-a219-4f38-9b10-774b733f73b8-kube-api-access-n66hq\") pod \"thanos-querier-654777ddcb-bxmmw\" (UID: \"6c12900c-a219-4f38-9b10-774b733f73b8\") " pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.442576 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.442535    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerStarted","Data":"d37f1a114887a1fe6ab9e9918ad5a932087a3f1c9dbc2a89a0c8e74e33e5e0f9"}
Jul 28 04:16:24.444794 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.444767    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb" event={"ID":"478a0703-2277-4856-aead-a0b3e7a01fc9","Type":"ContainerStarted","Data":"3c2926a1fc496698963c0b00cff56b12640add14b75d757f8148483c177be6fa"}
Jul 28 04:16:24.447111 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.447087    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d5m5q" event={"ID":"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc","Type":"ContainerStarted","Data":"fc8d615ebe11d9351dc6d24b0a66e5452b30f52e1ea4f98b203fce57bba2ab37"}
Jul 28 04:16:24.447257 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.447137    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d5m5q" event={"ID":"2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc","Type":"ContainerStarted","Data":"0bf1d0f7ab7ac0baa83ff95fcc607096cb15dbe6afa16b063aeb62e645b28595"}
Jul 28 04:16:24.460079 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.459992    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pkpjb" podStartSLOduration=2.416660331 podStartE2EDuration="3.459975105s" totalImagesPullingTime="1.043314774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:16:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:16:24.459046828 +0000 UTC m=+279.400886549" watchObservedRunningTime="2026-07-28 04:16:24.459975105 +0000 UTC m=+279.401814829"
Jul 28 04:16:24.477139 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.477103    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d5m5q" podStartSLOduration=2.229903495 podStartE2EDuration="3.477091179s" totalImagesPullingTime="1.187450301s" totalInitContainerRuntime="59.737383ms" isStatefulPod=true podCreationTimestamp="2026-07-28 04:16:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:16:24.47574693 +0000 UTC m=+279.417586650" watchObservedRunningTime="2026-07-28 04:16:24.477091179 +0000 UTC m=+279.418930898"
Jul 28 04:16:24.512991 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.512967    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:24.694800 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.694776    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-654777ddcb-bxmmw"]
Jul 28 04:16:24.697393 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:16:24.697351    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c12900c_a219_4f38_9b10_774b733f73b8.slice/crio-154853c19bacd7eba7984db2a54868c119e0798cbc5d3580958daa7d97833d1d WatchSource:0}: Error finding container 154853c19bacd7eba7984db2a54868c119e0798cbc5d3580958daa7d97833d1d: Status 404 returned error can't find the container with id 154853c19bacd7eba7984db2a54868c119e0798cbc5d3580958daa7d97833d1d
Jul 28 04:16:24.773013 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.772985    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:24.873254 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:24.873226    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:25.014580 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:25.014526    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:25.226906 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:25.226880    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:25.451326 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:25.451301    2574 generic.go:356] "Generic (PLEG): container finished" podID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerID="00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6" exitCode=0
Jul 28 04:16:25.451463 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:25.451380    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerDied","Data":"00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6"}
Jul 28 04:16:25.452724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:25.452696    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw" event={"ID":"6c12900c-a219-4f38-9b10-774b733f73b8","Type":"ContainerStarted","Data":"154853c19bacd7eba7984db2a54868c119e0798cbc5d3580958daa7d97833d1d"}
Jul 28 04:16:25.991966 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:25.991921    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:26.007924 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.007893    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"]
Jul 28 04:16:26.010931 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.010906    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"
Jul 28 04:16:26.012809 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.012777    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-frjvj\""
Jul 28 04:16:26.012913 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.012827    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 04:16:26.018482 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.018437    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"]
Jul 28 04:16:26.072802 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.072771    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5trrq\" (UniqueName: \"kubernetes.io/projected/9209391d-eaf3-4e2b-b501-e981beb2c808-kube-api-access-5trrq\") pod \"monitoring-plugin-68b7fc9574-c7fcv\" (UID: \"9209391d-eaf3-4e2b-b501-e981beb2c808\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"
Jul 28 04:16:26.072914 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.072813    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9209391d-eaf3-4e2b-b501-e981beb2c808-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-c7fcv\" (UID: \"9209391d-eaf3-4e2b-b501-e981beb2c808\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"
Jul 28 04:16:26.084326 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:16:26.084291    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vqdlc" podUID="3411355c-8caa-429b-88cc-97ddaabc7760"
Jul 28 04:16:26.089458 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.089435    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:26.173680 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.173655    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5trrq\" (UniqueName: \"kubernetes.io/projected/9209391d-eaf3-4e2b-b501-e981beb2c808-kube-api-access-5trrq\") pod \"monitoring-plugin-68b7fc9574-c7fcv\" (UID: \"9209391d-eaf3-4e2b-b501-e981beb2c808\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"
Jul 28 04:16:26.173788 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.173690    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9209391d-eaf3-4e2b-b501-e981beb2c808-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-c7fcv\" (UID: \"9209391d-eaf3-4e2b-b501-e981beb2c808\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"
Jul 28 04:16:26.173866 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:16:26.173851    2574 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 28 04:16:26.173915 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:16:26.173906    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9209391d-eaf3-4e2b-b501-e981beb2c808-monitoring-plugin-cert podName:9209391d-eaf3-4e2b-b501-e981beb2c808 nodeName:}" failed. No retries permitted until 2026-07-28 04:16:26.673889729 +0000 UTC m=+281.615729446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/9209391d-eaf3-4e2b-b501-e981beb2c808-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-c7fcv" (UID: "9209391d-eaf3-4e2b-b501-e981beb2c808") : secret "monitoring-plugin-cert" not found
Jul 28 04:16:26.181816 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.181792    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5trrq\" (UniqueName: \"kubernetes.io/projected/9209391d-eaf3-4e2b-b501-e981beb2c808-kube-api-access-5trrq\") pod \"monitoring-plugin-68b7fc9574-c7fcv\" (UID: \"9209391d-eaf3-4e2b-b501-e981beb2c808\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"
Jul 28 04:16:26.456276 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.456252    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:16:26.565239 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.565211    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:26.619794 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.619761    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:26.650079 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.650033    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw" podUID="1a8f78e5-c82f-4564-8f8b-43c779cf52db" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 28 04:16:26.678537 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.678450    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9209391d-eaf3-4e2b-b501-e981beb2c808-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-c7fcv\" (UID: \"9209391d-eaf3-4e2b-b501-e981beb2c808\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"
Jul 28 04:16:26.678648 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:16:26.678566    2574 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 28 04:16:26.678648 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:16:26.678624    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9209391d-eaf3-4e2b-b501-e981beb2c808-monitoring-plugin-cert podName:9209391d-eaf3-4e2b-b501-e981beb2c808 nodeName:}" failed. No retries permitted until 2026-07-28 04:16:27.678606549 +0000 UTC m=+282.620446250 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/9209391d-eaf3-4e2b-b501-e981beb2c808-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-c7fcv" (UID: "9209391d-eaf3-4e2b-b501-e981beb2c808") : secret "monitoring-plugin-cert" not found
Jul 28 04:16:26.812454 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:26.812415    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:27.188051 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:27.187968    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:27.289118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:27.289084    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:27.482212 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:27.482149    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:27.686765 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:27.686737    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9209391d-eaf3-4e2b-b501-e981beb2c808-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-c7fcv\" (UID: \"9209391d-eaf3-4e2b-b501-e981beb2c808\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"
Jul 28 04:16:27.689119 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:27.689090    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9209391d-eaf3-4e2b-b501-e981beb2c808-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-c7fcv\" (UID: \"9209391d-eaf3-4e2b-b501-e981beb2c808\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"
Jul 28 04:16:27.823229 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:27.823201    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"
Jul 28 04:16:28.009114 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:28.009091    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"]
Jul 28 04:16:28.011285 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:16:28.011256    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9209391d_eaf3_4e2b_b501_e981beb2c808.slice/crio-15a9dc612558661590523519b2479be944bb515d98f75f1bdf61beb7791e564c WatchSource:0}: Error finding container 15a9dc612558661590523519b2479be944bb515d98f75f1bdf61beb7791e564c: Status 404 returned error can't find the container with id 15a9dc612558661590523519b2479be944bb515d98f75f1bdf61beb7791e564c
Jul 28 04:16:28.466203 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:28.466174    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerStarted","Data":"41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd"}
Jul 28 04:16:28.466324 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:28.466239    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerStarted","Data":"79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba"}
Jul 28 04:16:28.466324 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:28.466273    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerStarted","Data":"657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e"}
Jul 28 04:16:28.466324 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:28.466298    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerStarted","Data":"932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a"}
Jul 28 04:16:28.466451 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:28.466324    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerStarted","Data":"677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb"}
Jul 28 04:16:28.468203 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:28.468182    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw" event={"ID":"6c12900c-a219-4f38-9b10-774b733f73b8","Type":"ContainerStarted","Data":"157f332d1e0666e71d9f4d76e3c64d08d3d31986d40e18bba4d14134937db639"}
Jul 28 04:16:28.468305 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:28.468249    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw" event={"ID":"6c12900c-a219-4f38-9b10-774b733f73b8","Type":"ContainerStarted","Data":"3e2d5ecc4c697b5e4e7e12e68b747b4105bb2e9408a48c2676f23e312e19f7ff"}
Jul 28 04:16:28.468305 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:28.468295    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw" event={"ID":"6c12900c-a219-4f38-9b10-774b733f73b8","Type":"ContainerStarted","Data":"3ac321687b6265436946e837964078a954bd5ea4b8179f56b2ef1a3d006ed007"}
Jul 28 04:16:28.469201 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:28.469180    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv" event={"ID":"9209391d-eaf3-4e2b-b501-e981beb2c808","Type":"ContainerStarted","Data":"15a9dc612558661590523519b2479be944bb515d98f75f1bdf61beb7791e564c"}
Jul 28 04:16:30.006836 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.006812    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:16:30.008849 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.008824    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3411355c-8caa-429b-88cc-97ddaabc7760-metrics-tls\") pod \"dns-default-vqdlc\" (UID: \"3411355c-8caa-429b-88cc-97ddaabc7760\") " pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:16:30.058616 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.058599    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7zgg8\""
Jul 28 04:16:30.067024 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.067002    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:16:30.324590 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.324568    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vqdlc"]
Jul 28 04:16:30.327154 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:16:30.327131    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3411355c_8caa_429b_88cc_97ddaabc7760.slice/crio-ba81fcf4aeef0bea6ab8beb3f5085a6ce0776f5f92ecdb5d09fcb296a643113f WatchSource:0}: Error finding container ba81fcf4aeef0bea6ab8beb3f5085a6ce0776f5f92ecdb5d09fcb296a643113f: Status 404 returned error can't find the container with id ba81fcf4aeef0bea6ab8beb3f5085a6ce0776f5f92ecdb5d09fcb296a643113f
Jul 28 04:16:30.477179 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.477105    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv" event={"ID":"9209391d-eaf3-4e2b-b501-e981beb2c808","Type":"ContainerStarted","Data":"fcd2702f18a1feb4e0a4d47d5de760e0bcee2cb762874191eba6c6240d164afe"}
Jul 28 04:16:30.477323 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.477300    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"
Jul 28 04:16:30.478426 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.478376    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vqdlc" event={"ID":"3411355c-8caa-429b-88cc-97ddaabc7760","Type":"ContainerStarted","Data":"ba81fcf4aeef0bea6ab8beb3f5085a6ce0776f5f92ecdb5d09fcb296a643113f"}
Jul 28 04:16:30.481967 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.481940    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerStarted","Data":"b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8"}
Jul 28 04:16:30.482957 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.482935    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv"
Jul 28 04:16:30.484895 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.484877    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw" event={"ID":"6c12900c-a219-4f38-9b10-774b733f73b8","Type":"ContainerStarted","Data":"b29c205fa6a1317209dd83082ce05fe3f450f64d75e8eb5885a12abe276ffd5f"}
Jul 28 04:16:30.485131 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.484936    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw" event={"ID":"6c12900c-a219-4f38-9b10-774b733f73b8","Type":"ContainerStarted","Data":"cb7034b28adae17e4e322a762a02d1eb802d94535b8dac671f13e4caf37ccd6b"}
Jul 28 04:16:30.485131 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.484982    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:30.485131 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.485017    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw" event={"ID":"6c12900c-a219-4f38-9b10-774b733f73b8","Type":"ContainerStarted","Data":"9401589e244b83f052c3e4d4632a2a5d5eed68d3c85e67f77a776d99296be0e0"}
Jul 28 04:16:30.491780 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.491745    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-c7fcv" podStartSLOduration=3.301845808 podStartE2EDuration="5.491732164s" totalImagesPullingTime="2.189886356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:16:30.490529031 +0000 UTC m=+285.432368757" watchObservedRunningTime="2026-07-28 04:16:30.491732164 +0000 UTC m=+285.433571884"
Jul 28 04:16:30.525602 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.525555    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.196180569 podStartE2EDuration="8.525536223s" totalImagesPullingTime="4.316505663s" totalInitContainerRuntime="12.849991ms" isStatefulPod=false podCreationTimestamp="2026-07-28 04:16:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:16:30.524705081 +0000 UTC m=+285.466544802" watchObservedRunningTime="2026-07-28 04:16:30.525536223 +0000 UTC m=+285.467375943"
Jul 28 04:16:30.546141 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:30.546098    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw" podStartSLOduration=1.706297951 podStartE2EDuration="6.546085281s" totalImagesPullingTime="4.83978733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:16:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:16:30.54487002 +0000 UTC m=+285.486709741" watchObservedRunningTime="2026-07-28 04:16:30.546085281 +0000 UTC m=+285.487925002"
Jul 28 04:16:32.493561 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:32.493530    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vqdlc" event={"ID":"3411355c-8caa-429b-88cc-97ddaabc7760","Type":"ContainerStarted","Data":"91ce2c780b73c221175b8f58f463036b2f984b34367b37fd0e623a1ee7d62cde"}
Jul 28 04:16:33.499748 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:33.499714    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vqdlc" event={"ID":"3411355c-8caa-429b-88cc-97ddaabc7760","Type":"ContainerStarted","Data":"389ba104f611b001af93123865c83babb59dfd0aac5677372ee21f6e518d7475"}
Jul 28 04:16:33.500096 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:33.499777    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:16:33.513831 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:33.513780    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vqdlc" podStartSLOduration=252.577311081 podStartE2EDuration="4m14.513762087s" totalImagesPullingTime="1.936451006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:12:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:16:33.513215991 +0000 UTC m=+288.455055713" watchObservedRunningTime="2026-07-28 04:16:33.513762087 +0000 UTC m=+288.455601811"
Jul 28 04:16:36.493836 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:36.493811    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-654777ddcb-bxmmw"
Jul 28 04:16:36.650670 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:36.650638    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw" podUID="1a8f78e5-c82f-4564-8f8b-43c779cf52db" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 28 04:16:38.445741 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:38.445709    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:38.546531 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:38.546488    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:41.100252 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:41.100216    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:41.200826 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:41.200796    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:41.838924 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:41.838890    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:43.506670 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:43.506641    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vqdlc"
Jul 28 04:16:45.447254 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:45.447221    2574 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 04:16:46.650542 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:46.650506    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw" podUID="1a8f78e5-c82f-4564-8f8b-43c779cf52db" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 28 04:16:46.652787 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:46.650569    2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw"
Jul 28 04:16:46.652787 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:46.650969    2574 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"9b34dcc44c0eae9207a845cbb7a72e8cf71104da4f573732f59492b8e4fe50a1"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 28 04:16:46.652787 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:46.651014    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw" podUID="1a8f78e5-c82f-4564-8f8b-43c779cf52db" containerName="service-proxy" containerID="cri-o://9b34dcc44c0eae9207a845cbb7a72e8cf71104da4f573732f59492b8e4fe50a1" gracePeriod=30
Jul 28 04:16:47.036751 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:47.036724    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:47.552831 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:47.552802    2574 generic.go:356] "Generic (PLEG): container finished" podID="1a8f78e5-c82f-4564-8f8b-43c779cf52db" containerID="9b34dcc44c0eae9207a845cbb7a72e8cf71104da4f573732f59492b8e4fe50a1" exitCode=2
Jul 28 04:16:47.552988 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:47.552840    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw" event={"ID":"1a8f78e5-c82f-4564-8f8b-43c779cf52db","Type":"ContainerDied","Data":"9b34dcc44c0eae9207a845cbb7a72e8cf71104da4f573732f59492b8e4fe50a1"}
Jul 28 04:16:47.552988 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:47.552869    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f596f584c-whrvw" event={"ID":"1a8f78e5-c82f-4564-8f8b-43c779cf52db","Type":"ContainerStarted","Data":"28710e8cbd9cf16ec052e89812e001f3a3e0a1fcece7cc841cc34265cf618ca8"}
Jul 28 04:16:48.153069 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:48.153035    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:48.253477 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:48.253448    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:48.354016 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:48.353991    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:48.454484 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:48.454433    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:48.504974 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:48.504942    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:49.637470 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:49.637438    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:49.684611 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:49.684586    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:50.244449 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:50.244426    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:50.634279 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:50.634254    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:50.683173 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:50.683140    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:56.900722 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:56.900690    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:56.947916 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:56.947886    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:57.017186 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:57.017161    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:57.063935 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:57.063908    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:57.223350 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:57.223299    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:57.323842 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:57.323808    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:58.471408 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:58.471355    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:16:58.571918 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:16:58.571891    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:01.129734 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:01.129701    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:01.227048 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:01.227021    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:07.545253 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:07.545222    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:07.594827 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:07.594803    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:13.138319 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:13.138271    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:13.225127 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:13.225078    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:16.124243 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:16.124212    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:16.177763 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:16.177737    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:22.634372 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:22.634338    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:22.733542 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:22.733518    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:22.792982 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:22.792958    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:33.609116 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:33.609084    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:33.709195 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:33.709139    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:34.240279 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:34.240251    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:34.288507 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:34.288482    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:40.411025 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:40.410943    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:40.815792 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:40.815768    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:41.465971 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.465938    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 04:17:41.466346 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.466204    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="alertmanager" containerID="cri-o://677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb" gracePeriod=120
Jul 28 04:17:41.466346 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.466273    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="kube-rbac-proxy-metric" containerID="cri-o://41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd" gracePeriod=120
Jul 28 04:17:41.466346 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.466317    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="kube-rbac-proxy-web" containerID="cri-o://657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e" gracePeriod=120
Jul 28 04:17:41.466553 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.466337    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="kube-rbac-proxy" containerID="cri-o://79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba" gracePeriod=120
Jul 28 04:17:41.466553 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.466373    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="config-reloader" containerID="cri-o://932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a" gracePeriod=120
Jul 28 04:17:41.466553 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.466398    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="prom-label-proxy" containerID="cri-o://b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8" gracePeriod=120
Jul 28 04:17:41.739233 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.738868    2574 generic.go:356] "Generic (PLEG): container finished" podID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerID="b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8" exitCode=0
Jul 28 04:17:41.739233 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.738894    2574 generic.go:356] "Generic (PLEG): container finished" podID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerID="79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba" exitCode=0
Jul 28 04:17:41.739233 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.738907    2574 generic.go:356] "Generic (PLEG): container finished" podID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerID="932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a" exitCode=0
Jul 28 04:17:41.739233 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.738916    2574 generic.go:356] "Generic (PLEG): container finished" podID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerID="677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb" exitCode=0
Jul 28 04:17:41.739233 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.739006    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerDied","Data":"b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8"}
Jul 28 04:17:41.739233 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.739069    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerDied","Data":"79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba"}
Jul 28 04:17:41.739233 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.739108    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerDied","Data":"932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a"}
Jul 28 04:17:41.739233 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:41.739144    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerDied","Data":"677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb"}
Jul 28 04:17:42.709612 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.709586    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:42.750897 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.750860    2574 generic.go:356] "Generic (PLEG): container finished" podID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerID="41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd" exitCode=0
Jul 28 04:17:42.750897 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.750894    2574 generic.go:356] "Generic (PLEG): container finished" podID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerID="657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e" exitCode=0
Jul 28 04:17:42.751083 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.750944    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerDied","Data":"41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd"}
Jul 28 04:17:42.751083 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.750989    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:42.751083 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.751054    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerDied","Data":"657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e"}
Jul 28 04:17:42.751222 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.751097    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"997996e2-1121-4647-8f6d-52dfd2ed897d","Type":"ContainerDied","Data":"d37f1a114887a1fe6ab9e9918ad5a932087a3f1c9dbc2a89a0c8e74e33e5e0f9"}
Jul 28 04:17:42.751957 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.751142    2574 scope.go:117] "RemoveContainer" containerID="b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8"
Jul 28 04:17:42.759571 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.759549    2574 scope.go:117] "RemoveContainer" containerID="41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd"
Jul 28 04:17:42.767261 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.767244    2574 scope.go:117] "RemoveContainer" containerID="79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba"
Jul 28 04:17:42.775422 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.775383    2574 scope.go:117] "RemoveContainer" containerID="657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e"
Jul 28 04:17:42.782409 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.782392    2574 scope.go:117] "RemoveContainer" containerID="932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a"
Jul 28 04:17:42.789017 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.788996    2574 scope.go:117] "RemoveContainer" containerID="677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb"
Jul 28 04:17:42.795577 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.795550    2574 scope.go:117] "RemoveContainer" containerID="00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6"
Jul 28 04:17:42.801741 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.801728    2574 scope.go:117] "RemoveContainer" containerID="b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8"
Jul 28 04:17:42.801974 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:17:42.801956    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8\": container with ID starting with b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8 not found: ID does not exist" containerID="b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8"
Jul 28 04:17:42.802028 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.801983    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8"} err="failed to get container status \"b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8\": rpc error: code = NotFound desc = could not find container \"b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8\": container with ID starting with b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8 not found: ID does not exist"
Jul 28 04:17:42.802028 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.802002    2574 scope.go:117] "RemoveContainer" containerID="41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd"
Jul 28 04:17:42.802243 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:17:42.802226    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd\": container with ID starting with 41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd not found: ID does not exist" containerID="41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd"
Jul 28 04:17:42.802282 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.802250    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd"} err="failed to get container status \"41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd\": rpc error: code = NotFound desc = could not find container \"41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd\": container with ID starting with 41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd not found: ID does not exist"
Jul 28 04:17:42.802282 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.802265    2574 scope.go:117] "RemoveContainer" containerID="79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba"
Jul 28 04:17:42.802506 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:17:42.802491    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba\": container with ID starting with 79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba not found: ID does not exist" containerID="79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba"
Jul 28 04:17:42.802550 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.802510    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba"} err="failed to get container status \"79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba\": rpc error: code = NotFound desc = could not find container \"79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba\": container with ID starting with 79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba not found: ID does not exist"
Jul 28 04:17:42.802550 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.802522    2574 scope.go:117] "RemoveContainer" containerID="657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e"
Jul 28 04:17:42.802724 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:17:42.802708    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e\": container with ID starting with 657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e not found: ID does not exist" containerID="657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e"
Jul 28 04:17:42.802794 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.802729    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e"} err="failed to get container status \"657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e\": rpc error: code = NotFound desc = could not find container \"657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e\": container with ID starting with 657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e not found: ID does not exist"
Jul 28 04:17:42.802794 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.802754    2574 scope.go:117] "RemoveContainer" containerID="932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a"
Jul 28 04:17:42.802970 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:17:42.802953    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a\": container with ID starting with 932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a not found: ID does not exist" containerID="932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a"
Jul 28 04:17:42.803014 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.802971    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a"} err="failed to get container status \"932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a\": rpc error: code = NotFound desc = could not find container \"932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a\": container with ID starting with 932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a not found: ID does not exist"
Jul 28 04:17:42.803014 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.802982    2574 scope.go:117] "RemoveContainer" containerID="677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb"
Jul 28 04:17:42.803168 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:17:42.803155    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb\": container with ID starting with 677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb not found: ID does not exist" containerID="677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb"
Jul 28 04:17:42.803209 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.803170    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb"} err="failed to get container status \"677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb\": rpc error: code = NotFound desc = could not find container \"677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb\": container with ID starting with 677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb not found: ID does not exist"
Jul 28 04:17:42.803209 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.803180    2574 scope.go:117] "RemoveContainer" containerID="00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6"
Jul 28 04:17:42.803390 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:17:42.803371    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6\": container with ID starting with 00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6 not found: ID does not exist" containerID="00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6"
Jul 28 04:17:42.803436 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.803394    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6"} err="failed to get container status \"00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6\": rpc error: code = NotFound desc = could not find container \"00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6\": container with ID starting with 00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6 not found: ID does not exist"
Jul 28 04:17:42.803436 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.803406    2574 scope.go:117] "RemoveContainer" containerID="b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8"
Jul 28 04:17:42.803619 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.803605    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8"} err="failed to get container status \"b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8\": rpc error: code = NotFound desc = could not find container \"b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8\": container with ID starting with b7e683f6ad38eeaa928457e99182db656b9f0d22ab568e642200c82be53647c8 not found: ID does not exist"
Jul 28 04:17:42.803656 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.803619    2574 scope.go:117] "RemoveContainer" containerID="41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd"
Jul 28 04:17:42.803806 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.803790    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd"} err="failed to get container status \"41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd\": rpc error: code = NotFound desc = could not find container \"41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd\": container with ID starting with 41f4ae322f7dfb99b03a7aac5b5d22a09204240066c59c0280713f3ff50252dd not found: ID does not exist"
Jul 28 04:17:42.803847 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.803806    2574 scope.go:117] "RemoveContainer" containerID="79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba"
Jul 28 04:17:42.803998 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.803984    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba"} err="failed to get container status \"79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba\": rpc error: code = NotFound desc = could not find container \"79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba\": container with ID starting with 79c01efddf96bd2ec10c253c503abfb0e2db9010e697ed6656cd7768fe151fba not found: ID does not exist"
Jul 28 04:17:42.804053 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.803997    2574 scope.go:117] "RemoveContainer" containerID="657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e"
Jul 28 04:17:42.804192 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.804177    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e"} err="failed to get container status \"657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e\": rpc error: code = NotFound desc = could not find container \"657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e\": container with ID starting with 657489277d62367d8b6bbda05ce3ee53638109e9d7eb8b3e22be111e002e2c2e not found: ID does not exist"
Jul 28 04:17:42.804192 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.804190    2574 scope.go:117] "RemoveContainer" containerID="932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a"
Jul 28 04:17:42.804420 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.804402    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a"} err="failed to get container status \"932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a\": rpc error: code = NotFound desc = could not find container \"932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a\": container with ID starting with 932e8259c10adb8fdc05b41b5ef15e4f31f356937f3aff6ce2626529ce2e066a not found: ID does not exist"
Jul 28 04:17:42.804471 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.804420    2574 scope.go:117] "RemoveContainer" containerID="677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb"
Jul 28 04:17:42.804639 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.804623    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb"} err="failed to get container status \"677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb\": rpc error: code = NotFound desc = could not find container \"677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb\": container with ID starting with 677ec5377fec2c14bf5a06791017fc5d04c562ee70d7a10b3e1fbd58534e4aeb not found: ID does not exist"
Jul 28 04:17:42.804683 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.804641    2574 scope.go:117] "RemoveContainer" containerID="00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6"
Jul 28 04:17:42.804861 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.804842    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6"} err="failed to get container status \"00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6\": rpc error: code = NotFound desc = could not find container \"00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6\": container with ID starting with 00252d00186f2a35c89a16ca56e5899e587906cceedf2a71dd59f7556f60cbf6 not found: ID does not exist"
Jul 28 04:17:42.809106 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809087    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"997996e2-1121-4647-8f6d-52dfd2ed897d\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") "
Jul 28 04:17:42.809165 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809119    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-metrics-client-ca\") pod \"997996e2-1121-4647-8f6d-52dfd2ed897d\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") "
Jul 28 04:17:42.809165 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809140    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy-web\") pod \"997996e2-1121-4647-8f6d-52dfd2ed897d\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") "
Jul 28 04:17:42.809228 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809162    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-config-volume\") pod \"997996e2-1121-4647-8f6d-52dfd2ed897d\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") "
Jul 28 04:17:42.809228 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809208    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-main-db\") pod \"997996e2-1121-4647-8f6d-52dfd2ed897d\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") "
Jul 28 04:17:42.809303 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809236    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-trusted-ca-bundle\") pod \"997996e2-1121-4647-8f6d-52dfd2ed897d\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") "
Jul 28 04:17:42.809303 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809262    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-main-tls\") pod \"997996e2-1121-4647-8f6d-52dfd2ed897d\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") "
Jul 28 04:17:42.809426 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809303    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/997996e2-1121-4647-8f6d-52dfd2ed897d-tls-assets\") pod \"997996e2-1121-4647-8f6d-52dfd2ed897d\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") "
Jul 28 04:17:42.809426 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809348    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zq5f\" (UniqueName: \"kubernetes.io/projected/997996e2-1121-4647-8f6d-52dfd2ed897d-kube-api-access-9zq5f\") pod \"997996e2-1121-4647-8f6d-52dfd2ed897d\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") "
Jul 28 04:17:42.809426 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809394    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/997996e2-1121-4647-8f6d-52dfd2ed897d-config-out\") pod \"997996e2-1121-4647-8f6d-52dfd2ed897d\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") "
Jul 28 04:17:42.809572 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809426    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-web-config\") pod \"997996e2-1121-4647-8f6d-52dfd2ed897d\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") "
Jul 28 04:17:42.809572 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809458    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-cluster-tls-config\") pod \"997996e2-1121-4647-8f6d-52dfd2ed897d\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") "
Jul 28 04:17:42.809572 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809499    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy\") pod \"997996e2-1121-4647-8f6d-52dfd2ed897d\" (UID: \"997996e2-1121-4647-8f6d-52dfd2ed897d\") "
Jul 28 04:17:42.809572 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809512    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "997996e2-1121-4647-8f6d-52dfd2ed897d" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 04:17:42.809752 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809585    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "997996e2-1121-4647-8f6d-52dfd2ed897d" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 04:17:42.809752 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809739    2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-main-db\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:17:42.809850 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809808    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-metrics-client-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:17:42.810215 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.809944    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "997996e2-1121-4647-8f6d-52dfd2ed897d" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 04:17:42.812072 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.812006    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "997996e2-1121-4647-8f6d-52dfd2ed897d" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 04:17:42.812321 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.812292    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "997996e2-1121-4647-8f6d-52dfd2ed897d" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 04:17:42.812420 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.812319    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "997996e2-1121-4647-8f6d-52dfd2ed897d" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 04:17:42.812871 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.812826    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-config-volume" (OuterVolumeSpecName: "config-volume") pod "997996e2-1121-4647-8f6d-52dfd2ed897d" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 04:17:42.812871 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.812830    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/997996e2-1121-4647-8f6d-52dfd2ed897d-config-out" (OuterVolumeSpecName: "config-out") pod "997996e2-1121-4647-8f6d-52dfd2ed897d" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 04:17:42.813122 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.813094    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/997996e2-1121-4647-8f6d-52dfd2ed897d-kube-api-access-9zq5f" (OuterVolumeSpecName: "kube-api-access-9zq5f") pod "997996e2-1121-4647-8f6d-52dfd2ed897d" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d"). InnerVolumeSpecName "kube-api-access-9zq5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 04:17:42.813430 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.813410    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/997996e2-1121-4647-8f6d-52dfd2ed897d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "997996e2-1121-4647-8f6d-52dfd2ed897d" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 04:17:42.813756 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.813732    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "997996e2-1121-4647-8f6d-52dfd2ed897d" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 04:17:42.815990 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.815941    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "997996e2-1121-4647-8f6d-52dfd2ed897d" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 04:17:42.821620 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.821602    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-web-config" (OuterVolumeSpecName: "web-config") pod "997996e2-1121-4647-8f6d-52dfd2ed897d" (UID: "997996e2-1121-4647-8f6d-52dfd2ed897d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 04:17:42.910780 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.910760    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zq5f\" (UniqueName: \"kubernetes.io/projected/997996e2-1121-4647-8f6d-52dfd2ed897d-kube-api-access-9zq5f\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:17:42.910780 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.910779    2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/997996e2-1121-4647-8f6d-52dfd2ed897d-config-out\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:17:42.910871 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.910805    2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-web-config\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:17:42.910871 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.910814    2574 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-cluster-tls-config\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:17:42.910871 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.910823    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:17:42.910871 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.910833    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:17:42.910871 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.910843    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:17:42.910871 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.910851    2574 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-config-volume\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:17:42.910871 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.910859    2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/997996e2-1121-4647-8f6d-52dfd2ed897d-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:17:42.910871 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.910868    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/997996e2-1121-4647-8f6d-52dfd2ed897d-secret-alertmanager-main-tls\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:17:42.911092 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:42.910877    2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/997996e2-1121-4647-8f6d-52dfd2ed897d-tls-assets\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:17:43.073036 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.073013    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 04:17:43.076690 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.076670    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 04:17:43.098941 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.098918    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 04:17:43.099171 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099159    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="config-reloader"
Jul 28 04:17:43.099221 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099174    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="config-reloader"
Jul 28 04:17:43.099221 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099183    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="kube-rbac-proxy"
Jul 28 04:17:43.099221 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099189    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="kube-rbac-proxy"
Jul 28 04:17:43.099221 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099196    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="kube-rbac-proxy-metric"
Jul 28 04:17:43.099221 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099202    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="kube-rbac-proxy-metric"
Jul 28 04:17:43.099221 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099210    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="init-config-reloader"
Jul 28 04:17:43.099221 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099216    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="init-config-reloader"
Jul 28 04:17:43.099221 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099223    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="alertmanager"
Jul 28 04:17:43.099497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099228    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="alertmanager"
Jul 28 04:17:43.099497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099241    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="prom-label-proxy"
Jul 28 04:17:43.099497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099247    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="prom-label-proxy"
Jul 28 04:17:43.099497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099269    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="kube-rbac-proxy-web"
Jul 28 04:17:43.099497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099274    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="kube-rbac-proxy-web"
Jul 28 04:17:43.099497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099313    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="kube-rbac-proxy-metric"
Jul 28 04:17:43.099497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099319    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="config-reloader"
Jul 28 04:17:43.099497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099325    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="kube-rbac-proxy"
Jul 28 04:17:43.099497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099331    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="alertmanager"
Jul 28 04:17:43.099497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099336    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="prom-label-proxy"
Jul 28 04:17:43.099497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.099342    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" containerName="kube-rbac-proxy-web"
Jul 28 04:17:43.104216 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.104201    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.106082 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.106064    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 04:17:43.106425 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.106404    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 04:17:43.106527 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.106435    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 04:17:43.106527 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.106516    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 04:17:43.106692 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.106589    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 04:17:43.106692 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.106671    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nx68t\""
Jul 28 04:17:43.106826 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.106704    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 04:17:43.106826 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.106793    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 04:17:43.107064 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.106861    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 04:17:43.110651 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.110632    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 04:17:43.115154 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.115136    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 04:17:43.212934 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.212916    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/559a3031-9899-489c-b0b6-2012a7e14de5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.213044 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.212948    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-web-config\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.213044 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.212968    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.213044 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.212991    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/559a3031-9899-489c-b0b6-2012a7e14de5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.213224 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.213060    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw9g8\" (UniqueName: \"kubernetes.io/projected/559a3031-9899-489c-b0b6-2012a7e14de5-kube-api-access-vw9g8\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.213224 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.213096    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/559a3031-9899-489c-b0b6-2012a7e14de5-config-out\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.213224 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.213123    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/559a3031-9899-489c-b0b6-2012a7e14de5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.213224 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.213151    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.213224 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.213178    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-config-volume\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.213224 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.213206    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.213461 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.213228    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/559a3031-9899-489c-b0b6-2012a7e14de5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.213461 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.213261    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.213461 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.213309    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.313619 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.313566    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-web-config\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.313619 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.313593    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.313619 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.313614    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/559a3031-9899-489c-b0b6-2012a7e14de5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.313816 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.313630    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw9g8\" (UniqueName: \"kubernetes.io/projected/559a3031-9899-489c-b0b6-2012a7e14de5-kube-api-access-vw9g8\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.313816 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.313768    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/559a3031-9899-489c-b0b6-2012a7e14de5-config-out\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.313816 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.313806    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/559a3031-9899-489c-b0b6-2012a7e14de5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.313952 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.313834    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.313952 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.313863    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-config-volume\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.313952 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.313893    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.313952 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.313916    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/559a3031-9899-489c-b0b6-2012a7e14de5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.313952 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.313943    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.314196 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.313983    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.314196 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.314031    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/559a3031-9899-489c-b0b6-2012a7e14de5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.314196 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.314031    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/559a3031-9899-489c-b0b6-2012a7e14de5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.316103 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.316071    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-web-config\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.316448 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.316420    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/559a3031-9899-489c-b0b6-2012a7e14de5-config-out\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.316536 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.316512    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.316721 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.316701    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.316821 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.316798    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.316929 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.316898    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.317018 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.316998    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-config-volume\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.317312 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.317291    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/559a3031-9899-489c-b0b6-2012a7e14de5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.317458 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.317442    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/559a3031-9899-489c-b0b6-2012a7e14de5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.317807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.317783    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/559a3031-9899-489c-b0b6-2012a7e14de5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.318070 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.318055    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/559a3031-9899-489c-b0b6-2012a7e14de5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.320618 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.320600    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw9g8\" (UniqueName: \"kubernetes.io/projected/559a3031-9899-489c-b0b6-2012a7e14de5-kube-api-access-vw9g8\") pod \"alertmanager-main-0\" (UID: \"559a3031-9899-489c-b0b6-2012a7e14de5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.413897 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.413875    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 04:17:43.540451 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.540425    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 04:17:43.543114 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:17:43.543085    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod559a3031_9899_489c_b0b6_2012a7e14de5.slice/crio-0792d6bfe53c86ae8bccea1bc9000c2f40b48a13fae6691f8de75fa9c6c9b92b WatchSource:0}: Error finding container 0792d6bfe53c86ae8bccea1bc9000c2f40b48a13fae6691f8de75fa9c6c9b92b: Status 404 returned error can't find the container with id 0792d6bfe53c86ae8bccea1bc9000c2f40b48a13fae6691f8de75fa9c6c9b92b
Jul 28 04:17:43.619328 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.619304    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="997996e2-1121-4647-8f6d-52dfd2ed897d" path="/var/lib/kubelet/pods/997996e2-1121-4647-8f6d-52dfd2ed897d/volumes"
Jul 28 04:17:43.755977 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.755950    2574 generic.go:356] "Generic (PLEG): container finished" podID="559a3031-9899-489c-b0b6-2012a7e14de5" containerID="35a39e588b837e63988374db75697e1dd083d422ca1364302b91d71c91ea45be" exitCode=0
Jul 28 04:17:43.756317 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.756039    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"559a3031-9899-489c-b0b6-2012a7e14de5","Type":"ContainerDied","Data":"35a39e588b837e63988374db75697e1dd083d422ca1364302b91d71c91ea45be"}
Jul 28 04:17:43.756317 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:43.756113    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"559a3031-9899-489c-b0b6-2012a7e14de5","Type":"ContainerStarted","Data":"0792d6bfe53c86ae8bccea1bc9000c2f40b48a13fae6691f8de75fa9c6c9b92b"}
Jul 28 04:17:44.764541 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:44.764494    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"559a3031-9899-489c-b0b6-2012a7e14de5","Type":"ContainerStarted","Data":"fb149e338ce5081b5cd32f94058f9b9f75c36100550356848c2af1601a62a1bc"}
Jul 28 04:17:44.765011 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:44.764612    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"559a3031-9899-489c-b0b6-2012a7e14de5","Type":"ContainerStarted","Data":"e5c6779aa854af6698c9d09cc8474a3969837f20a5f9b43d3abaa858dd195091"}
Jul 28 04:17:44.765011 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:44.764657    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"559a3031-9899-489c-b0b6-2012a7e14de5","Type":"ContainerStarted","Data":"0191be6ec5227014b3a23aa8bb215da0dd28fa199b1fbc00323574f6ef2f15c0"}
Jul 28 04:17:44.765011 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:44.764703    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"559a3031-9899-489c-b0b6-2012a7e14de5","Type":"ContainerStarted","Data":"03ec0b15c7ed30dcc55f06dc3ee962622fd570a536bd8f853d165657da4ba316"}
Jul 28 04:17:44.765011 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:44.764742    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"559a3031-9899-489c-b0b6-2012a7e14de5","Type":"ContainerStarted","Data":"b3f094a32779ae166bc1128ac5c96be116de721e0cbc124aa9116efdb49d7109"}
Jul 28 04:17:44.765011 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:44.764787    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"559a3031-9899-489c-b0b6-2012a7e14de5","Type":"ContainerStarted","Data":"38a33405c51ac7bcf47652e9d6c737bd47b19de119ffa5e91ae756f1f3ed034b"}
Jul 28 04:17:44.788372 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:44.788308    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.771779182 podStartE2EDuration="1.788279079s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.499897ms" isStatefulPod=false podCreationTimestamp="2026-07-28 04:17:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:17:44.787613301 +0000 UTC m=+359.729453021" watchObservedRunningTime="2026-07-28 04:17:44.788279079 +0000 UTC m=+359.730118788"
Jul 28 04:17:45.495335 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.495303    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-55cc884f86-9rwmx"]
Jul 28 04:17:45.499953 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.499937    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.501830 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.501806    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 04:17:45.501962 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.501896    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-djzds\""
Jul 28 04:17:45.502110 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.502096    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 04:17:45.502160 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.502135    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 04:17:45.502194 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.502164    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 04:17:45.502194 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.502165    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 04:17:45.507310 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.507075    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 04:17:45.511570 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.511551    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55cc884f86-9rwmx"]
Jul 28 04:17:45.632244 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.632209    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/64a49d09-7679-4108-a2b3-2645497e4695-federate-client-tls\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.632418 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.632256    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a49d09-7679-4108-a2b3-2645497e4695-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.632418 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.632312    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/64a49d09-7679-4108-a2b3-2645497e4695-secret-telemeter-client\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.632418 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.632386    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64a49d09-7679-4108-a2b3-2645497e4695-metrics-client-ca\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.632418 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.632407    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a49d09-7679-4108-a2b3-2645497e4695-serving-certs-ca-bundle\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.632557 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.632447    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v95f8\" (UniqueName: \"kubernetes.io/projected/64a49d09-7679-4108-a2b3-2645497e4695-kube-api-access-v95f8\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.632557 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.632470    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/64a49d09-7679-4108-a2b3-2645497e4695-telemeter-client-tls\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.632557 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.632485    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64a49d09-7679-4108-a2b3-2645497e4695-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.733391 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.733299    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/64a49d09-7679-4108-a2b3-2645497e4695-federate-client-tls\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.733391 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.733378    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a49d09-7679-4108-a2b3-2645497e4695-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.733536 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.733413    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/64a49d09-7679-4108-a2b3-2645497e4695-secret-telemeter-client\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.733536 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.733454    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64a49d09-7679-4108-a2b3-2645497e4695-metrics-client-ca\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.733536 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.733472    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a49d09-7679-4108-a2b3-2645497e4695-serving-certs-ca-bundle\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.733536 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.733531    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v95f8\" (UniqueName: \"kubernetes.io/projected/64a49d09-7679-4108-a2b3-2645497e4695-kube-api-access-v95f8\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.733688 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.733566    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/64a49d09-7679-4108-a2b3-2645497e4695-telemeter-client-tls\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.733688 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.733588    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64a49d09-7679-4108-a2b3-2645497e4695-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.734663 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.734623    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64a49d09-7679-4108-a2b3-2645497e4695-metrics-client-ca\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.734932 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.734907    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a49d09-7679-4108-a2b3-2645497e4695-serving-certs-ca-bundle\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.735297 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.735273    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a49d09-7679-4108-a2b3-2645497e4695-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.736558 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.736528    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64a49d09-7679-4108-a2b3-2645497e4695-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.736663 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.736606    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/64a49d09-7679-4108-a2b3-2645497e4695-secret-telemeter-client\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.736765 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.736743    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/64a49d09-7679-4108-a2b3-2645497e4695-telemeter-client-tls\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.736815 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.736748    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/64a49d09-7679-4108-a2b3-2645497e4695-federate-client-tls\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.740204 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.740188    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v95f8\" (UniqueName: \"kubernetes.io/projected/64a49d09-7679-4108-a2b3-2645497e4695-kube-api-access-v95f8\") pod \"telemeter-client-55cc884f86-9rwmx\" (UID: \"64a49d09-7679-4108-a2b3-2645497e4695\") " pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.810435 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.810406    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx"
Jul 28 04:17:45.936289 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.936081    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55cc884f86-9rwmx"]
Jul 28 04:17:45.938290 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:17:45.938264    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a49d09_7679_4108_a2b3_2645497e4695.slice/crio-834da21ae8d7171a320b4e4a1568ce2ecb9ec8fd0379b1d9d2beeebe542a9155 WatchSource:0}: Error finding container 834da21ae8d7171a320b4e4a1568ce2ecb9ec8fd0379b1d9d2beeebe542a9155: Status 404 returned error can't find the container with id 834da21ae8d7171a320b4e4a1568ce2ecb9ec8fd0379b1d9d2beeebe542a9155
Jul 28 04:17:45.940165 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:45.940149    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 04:17:46.774302 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:46.774265    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx" event={"ID":"64a49d09-7679-4108-a2b3-2645497e4695","Type":"ContainerStarted","Data":"834da21ae8d7171a320b4e4a1568ce2ecb9ec8fd0379b1d9d2beeebe542a9155"}
Jul 28 04:17:48.783553 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:48.783518    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx" event={"ID":"64a49d09-7679-4108-a2b3-2645497e4695","Type":"ContainerStarted","Data":"5b33f48b6d5d04bbed700e322ca3fb307a24b0267f3d457a851f9bfe90337ea2"}
Jul 28 04:17:48.783930 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:48.783573    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx" event={"ID":"64a49d09-7679-4108-a2b3-2645497e4695","Type":"ContainerStarted","Data":"dd5975fbcefebaa01020f9b435c7bc204fbd87a883cf7bcd6043aea82023a359"}
Jul 28 04:17:48.783930 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:48.783592    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx" event={"ID":"64a49d09-7679-4108-a2b3-2645497e4695","Type":"ContainerStarted","Data":"b1e7e9d022e39ea6b136ebc13a33fd815d6bcefc44f1ec7952ae228d962d310a"}
Jul 28 04:17:48.803321 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:48.803271    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-55cc884f86-9rwmx" podStartSLOduration=1.371545387 podStartE2EDuration="3.80325702s" totalImagesPullingTime="2.431711633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:17:48.801084094 +0000 UTC m=+363.742923825" watchObservedRunningTime="2026-07-28 04:17:48.80325702 +0000 UTC m=+363.745096739"
Jul 28 04:17:49.747752 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:49.747722    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:49.848250 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:49.848221    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:55.572966 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:55.572931    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:17:55.623646 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:17:55.623617    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:18:00.270426 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:18:00.270388    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:18:00.322594 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:18:00.322565    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:18:00.373381 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:18:00.373338    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:18:01.121010 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:18:01.120971    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:18:01.221021 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:18:01.220982    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:18:30.025421 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:18:30.025394    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:18:30.072689 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:18:30.072669    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:18:55.635680 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:18:55.635648    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:18:55.690328 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:18:55.690300    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:18:59.330712 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:18:59.330678    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:18:59.380960 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:18:59.380933    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:18:59.476214 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:18:59.476188    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:18:59.535115 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:18:59.535090    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:19:00.079082 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:19:00.079046    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:19:07.582290 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:19:07.582214    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:19:07.634126 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:19:07.634102    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:19:10.524946 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:19:10.524912    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:19:10.572619 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:19:10.572585    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:19:10.625106 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:19:10.625086    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:19:16.132110 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:19:16.132073    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:19:16.234283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:19:16.234257    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:19:47.007660 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:19:47.007626    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:19:55.528781 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:19:55.528742    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:16.073813 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:16.073781    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:16.174670 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:16.174632    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:42.613072 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:42.612999    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:43.409814 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:43.409784    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:46.213074 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:46.213044    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:46.265147 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:46.265124    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:47.720981 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:47.720948    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:47.825928 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:47.825866    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:48.118024 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:48.117993    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:48.245587 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:48.245541    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:49.932159 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:49.932114    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:50.113835 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:50.113806    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:53.013212 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:53.013132    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:53.068778 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:53.068746    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:55.709090 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:55.709057    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:55.763797 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:55.763771    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:58.334910 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.334878    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:20:58.384165 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.384137    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rdbng"]
Jul 28 04:20:58.387098 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.387083    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rdbng"
Jul 28 04:20:58.389027 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.389007    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 04:20:58.389116 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.389087    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 04:20:58.389209 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.389194    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-q4g47\""
Jul 28 04:20:58.394048 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.394026    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rdbng"]
Jul 28 04:20:58.543916 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.543890    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d390e0a-1b44-4f8b-ba68-d82de94ff417-bound-sa-token\") pod \"cert-manager-55b4d954d5-rdbng\" (UID: \"3d390e0a-1b44-4f8b-ba68-d82de94ff417\") " pod="cert-manager/cert-manager-55b4d954d5-rdbng"
Jul 28 04:20:58.544061 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.543929    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xncvk\" (UniqueName: \"kubernetes.io/projected/3d390e0a-1b44-4f8b-ba68-d82de94ff417-kube-api-access-xncvk\") pod \"cert-manager-55b4d954d5-rdbng\" (UID: \"3d390e0a-1b44-4f8b-ba68-d82de94ff417\") " pod="cert-manager/cert-manager-55b4d954d5-rdbng"
Jul 28 04:20:58.645185 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.645115    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d390e0a-1b44-4f8b-ba68-d82de94ff417-bound-sa-token\") pod \"cert-manager-55b4d954d5-rdbng\" (UID: \"3d390e0a-1b44-4f8b-ba68-d82de94ff417\") " pod="cert-manager/cert-manager-55b4d954d5-rdbng"
Jul 28 04:20:58.645185 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.645166    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xncvk\" (UniqueName: \"kubernetes.io/projected/3d390e0a-1b44-4f8b-ba68-d82de94ff417-kube-api-access-xncvk\") pod \"cert-manager-55b4d954d5-rdbng\" (UID: \"3d390e0a-1b44-4f8b-ba68-d82de94ff417\") " pod="cert-manager/cert-manager-55b4d954d5-rdbng"
Jul 28 04:20:58.652079 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.652044    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d390e0a-1b44-4f8b-ba68-d82de94ff417-bound-sa-token\") pod \"cert-manager-55b4d954d5-rdbng\" (UID: \"3d390e0a-1b44-4f8b-ba68-d82de94ff417\") " pod="cert-manager/cert-manager-55b4d954d5-rdbng"
Jul 28 04:20:58.652198 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.652182    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xncvk\" (UniqueName: \"kubernetes.io/projected/3d390e0a-1b44-4f8b-ba68-d82de94ff417-kube-api-access-xncvk\") pod \"cert-manager-55b4d954d5-rdbng\" (UID: \"3d390e0a-1b44-4f8b-ba68-d82de94ff417\") " pod="cert-manager/cert-manager-55b4d954d5-rdbng"
Jul 28 04:20:58.696978 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.696956    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rdbng"
Jul 28 04:20:58.815522 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:58.815477    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rdbng"]
Jul 28 04:20:58.819893 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:20:58.819867    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d390e0a_1b44_4f8b_ba68_d82de94ff417.slice/crio-682d72386f1a7e2818a7f028a33fabb74a1d6d5a82ca0be015456853a882ff79 WatchSource:0}: Error finding container 682d72386f1a7e2818a7f028a33fabb74a1d6d5a82ca0be015456853a882ff79: Status 404 returned error can't find the container with id 682d72386f1a7e2818a7f028a33fabb74a1d6d5a82ca0be015456853a882ff79
Jul 28 04:20:59.480561 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:20:59.480506    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rdbng" event={"ID":"3d390e0a-1b44-4f8b-ba68-d82de94ff417","Type":"ContainerStarted","Data":"682d72386f1a7e2818a7f028a33fabb74a1d6d5a82ca0be015456853a882ff79"}
Jul 28 04:21:00.481827 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:00.481794    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:00.579505 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:00.579473    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:02.187627 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.187595    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx"]
Jul 28 04:21:02.190739 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.190723    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx"
Jul 28 04:21:02.192683 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.192655    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-x78mg\""
Jul 28 04:21:02.192683 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.192670    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 04:21:02.192683 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.192656    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 04:21:02.199981 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.199960    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx"]
Jul 28 04:21:02.275003 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.274982    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqb8z\" (UniqueName: \"kubernetes.io/projected/067c7e82-47a0-4d63-8033-52625050f420-kube-api-access-bqb8z\") pod \"openshift-lws-operator-fd8ccff4c-v74fx\" (UID: \"067c7e82-47a0-4d63-8033-52625050f420\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx"
Jul 28 04:21:02.275125 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.275042    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/067c7e82-47a0-4d63-8033-52625050f420-tmp\") pod \"openshift-lws-operator-fd8ccff4c-v74fx\" (UID: \"067c7e82-47a0-4d63-8033-52625050f420\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx"
Jul 28 04:21:02.376015 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.375995    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqb8z\" (UniqueName: \"kubernetes.io/projected/067c7e82-47a0-4d63-8033-52625050f420-kube-api-access-bqb8z\") pod \"openshift-lws-operator-fd8ccff4c-v74fx\" (UID: \"067c7e82-47a0-4d63-8033-52625050f420\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx"
Jul 28 04:21:02.376130 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.376059    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/067c7e82-47a0-4d63-8033-52625050f420-tmp\") pod \"openshift-lws-operator-fd8ccff4c-v74fx\" (UID: \"067c7e82-47a0-4d63-8033-52625050f420\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx"
Jul 28 04:21:02.376389 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.376354    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/067c7e82-47a0-4d63-8033-52625050f420-tmp\") pod \"openshift-lws-operator-fd8ccff4c-v74fx\" (UID: \"067c7e82-47a0-4d63-8033-52625050f420\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx"
Jul 28 04:21:02.382902 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.382880    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqb8z\" (UniqueName: \"kubernetes.io/projected/067c7e82-47a0-4d63-8033-52625050f420-kube-api-access-bqb8z\") pod \"openshift-lws-operator-fd8ccff4c-v74fx\" (UID: \"067c7e82-47a0-4d63-8033-52625050f420\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx"
Jul 28 04:21:02.495458 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.495387    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rdbng" event={"ID":"3d390e0a-1b44-4f8b-ba68-d82de94ff417","Type":"ContainerStarted","Data":"0d201666c8fd677014fa96d98597bda8d8b2c6afb911204e85bf53dfeaf2866d"}
Jul 28 04:21:02.499633 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.499617    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx"
Jul 28 04:21:02.510580 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.510532    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rdbng" podStartSLOduration=1.6814262800000002 podStartE2EDuration="4.510514566s" totalImagesPullingTime="2.829088286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:21:02.509425427 +0000 UTC m=+557.451265148" watchObservedRunningTime="2026-07-28 04:21:02.510514566 +0000 UTC m=+557.452354288"
Jul 28 04:21:02.623027 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:02.622885    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx"]
Jul 28 04:21:02.625608 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:21:02.625585    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod067c7e82_47a0_4d63_8033_52625050f420.slice/crio-5345ab98253f83f7261ee96e6ae1df339991579563c8c579ba6962267f2a02c5 WatchSource:0}: Error finding container 5345ab98253f83f7261ee96e6ae1df339991579563c8c579ba6962267f2a02c5: Status 404 returned error can't find the container with id 5345ab98253f83f7261ee96e6ae1df339991579563c8c579ba6962267f2a02c5
Jul 28 04:21:03.502841 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:03.502803    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx" event={"ID":"067c7e82-47a0-4d63-8033-52625050f420","Type":"ContainerStarted","Data":"5345ab98253f83f7261ee96e6ae1df339991579563c8c579ba6962267f2a02c5"}
Jul 28 04:21:05.513494 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:05.513455    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx" event={"ID":"067c7e82-47a0-4d63-8033-52625050f420","Type":"ContainerStarted","Data":"f1383bb7490afeef23c663bcf872363a4f52450f191acbd48ea468be0ff13012"}
Jul 28 04:21:05.531074 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:05.531025    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-v74fx" podStartSLOduration=1.251481461 podStartE2EDuration="3.531011156s" totalImagesPullingTime="2.279529695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:21:05.529352725 +0000 UTC m=+560.471192464" watchObservedRunningTime="2026-07-28 04:21:05.531011156 +0000 UTC m=+560.472850876"
Jul 28 04:21:06.328984 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:06.328954    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:06.429885 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:06.429855    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:08.019681 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:08.019649    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:21.269556 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:21.269520    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:21.359230 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:21.359205    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:21.871995 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:21.871964    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:22.109750 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.109719    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:22.208793 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.208733    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:22.242020 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.241996    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"]
Jul 28 04:21:22.248309 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.248290    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.250427 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.250404    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 04:21:22.250524 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.250394    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 04:21:22.250524 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.250439    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9c6v4\""
Jul 28 04:21:22.250640 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.250576    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 04:21:22.250782 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.250767    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 04:21:22.263036 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.263015    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"]
Jul 28 04:21:22.310289 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.310263    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f5167944-00ac-41a7-9ac6-007f32e9d34c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.310642 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.310312    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5167944-00ac-41a7-9ac6-007f32e9d34c-webhook-cert\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.310642 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.310380    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5167944-00ac-41a7-9ac6-007f32e9d34c-tmp\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.310642 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.310417    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8qp2\" (UniqueName: \"kubernetes.io/projected/f5167944-00ac-41a7-9ac6-007f32e9d34c-kube-api-access-p8qp2\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.310642 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.310456    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f5167944-00ac-41a7-9ac6-007f32e9d34c-manifests\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.369003 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.368982    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:22.411439 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.411416    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5167944-00ac-41a7-9ac6-007f32e9d34c-webhook-cert\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.411553 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.411460    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5167944-00ac-41a7-9ac6-007f32e9d34c-tmp\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.411553 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.411494    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8qp2\" (UniqueName: \"kubernetes.io/projected/f5167944-00ac-41a7-9ac6-007f32e9d34c-kube-api-access-p8qp2\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.411553 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.411532    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f5167944-00ac-41a7-9ac6-007f32e9d34c-manifests\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.411707 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.411591    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f5167944-00ac-41a7-9ac6-007f32e9d34c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.411961 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.411935    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f5167944-00ac-41a7-9ac6-007f32e9d34c-manifests\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.413749 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.413727    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5167944-00ac-41a7-9ac6-007f32e9d34c-tmp\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.414008 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.413988    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5167944-00ac-41a7-9ac6-007f32e9d34c-webhook-cert\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.414049 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.413996    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f5167944-00ac-41a7-9ac6-007f32e9d34c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.420026 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.420005    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8qp2\" (UniqueName: \"kubernetes.io/projected/f5167944-00ac-41a7-9ac6-007f32e9d34c-kube-api-access-p8qp2\") pod \"opendatahub-operator-controller-manager-69d65f9dd8-kd8fw\" (UID: \"f5167944-00ac-41a7-9ac6-007f32e9d34c\") " pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.458276 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.458249    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:22.558173 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.558155    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:22.570587 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.570555    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:22.661514 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.661487    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:22.688083 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.688063    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"]
Jul 28 04:21:22.690346 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:21:22.690321    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5167944_00ac_41a7_9ac6_007f32e9d34c.slice/crio-bb86a28d0b1a87a0a3822196fa5015641b06ceee855713886c1a0484368e9d68 WatchSource:0}: Error finding container bb86a28d0b1a87a0a3822196fa5015641b06ceee855713886c1a0484368e9d68: Status 404 returned error can't find the container with id bb86a28d0b1a87a0a3822196fa5015641b06ceee855713886c1a0484368e9d68
Jul 28 04:21:22.760419 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.760391    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:22.860205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.860143    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:22.961229 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:22.961202    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:23.583687 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:23.583651    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw" event={"ID":"f5167944-00ac-41a7-9ac6-007f32e9d34c","Type":"ContainerStarted","Data":"bb86a28d0b1a87a0a3822196fa5015641b06ceee855713886c1a0484368e9d68"}
Jul 28 04:21:25.593771 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:25.593735    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw" event={"ID":"f5167944-00ac-41a7-9ac6-007f32e9d34c","Type":"ContainerStarted","Data":"7f8a6499c51bcef0e0214f75a4eaa8ae870eaff28e0c4094aa563395030b8e84"}
Jul 28 04:21:26.599679 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:26.599647    2574 generic.go:356] "Generic (PLEG): container finished" podID="f5167944-00ac-41a7-9ac6-007f32e9d34c" containerID="7f8a6499c51bcef0e0214f75a4eaa8ae870eaff28e0c4094aa563395030b8e84" exitCode=0
Jul 28 04:21:26.600106 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:26.599737    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw" event={"ID":"f5167944-00ac-41a7-9ac6-007f32e9d34c","Type":"ContainerDied","Data":"7f8a6499c51bcef0e0214f75a4eaa8ae870eaff28e0c4094aa563395030b8e84"}
Jul 28 04:21:27.605868 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:27.605836    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw" event={"ID":"f5167944-00ac-41a7-9ac6-007f32e9d34c","Type":"ContainerStarted","Data":"4716da18c4a4b09b3a000fe52a812628c313ba5fa896b563d64836631d03214a"}
Jul 28 04:21:27.606298 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:27.605904    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:27.624617 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:27.624572    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw" podStartSLOduration=1.556091131 podStartE2EDuration="5.624556443s" totalImagesPullingTime="3.124013532s" totalInitContainerRuntime="944.45178ms" isStatefulPod=false podCreationTimestamp="2026-07-28 04:21:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:21:27.622686525 +0000 UTC m=+582.564526244" watchObservedRunningTime="2026-07-28 04:21:27.624556443 +0000 UTC m=+582.566396209"
Jul 28 04:21:34.054964 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:34.054914    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:34.150222 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:34.150192    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:34.338251 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:34.338149    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:34.433777 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:34.433742    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:38.612711 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:38.612671    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-69d65f9dd8-kd8fw"
Jul 28 04:21:40.931333 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:40.931307    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:41.035645 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.035603    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:41.591515 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.591485    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pgzfs"]
Jul 28 04:21:41.594306 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.594290    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs"
Jul 28 04:21:41.596075 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.596044    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 04:21:41.596212 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.596195    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-sx9jg\""
Jul 28 04:21:41.601990 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.601969    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pgzfs"]
Jul 28 04:21:41.619734 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.619711    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:41.662970 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.662937    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf8kj\" (UniqueName: \"kubernetes.io/projected/4b3ebb7d-856f-4e16-9e04-786a216dee39-kube-api-access-qf8kj\") pod \"odh-model-controller-7998c67dc4-pgzfs\" (UID: \"4b3ebb7d-856f-4e16-9e04-786a216dee39\") " pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs"
Jul 28 04:21:41.663109 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.662991    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b3ebb7d-856f-4e16-9e04-786a216dee39-cert\") pod \"odh-model-controller-7998c67dc4-pgzfs\" (UID: \"4b3ebb7d-856f-4e16-9e04-786a216dee39\") " pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs"
Jul 28 04:21:41.719642 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.719614    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:41.763766 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.763727    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf8kj\" (UniqueName: \"kubernetes.io/projected/4b3ebb7d-856f-4e16-9e04-786a216dee39-kube-api-access-qf8kj\") pod \"odh-model-controller-7998c67dc4-pgzfs\" (UID: \"4b3ebb7d-856f-4e16-9e04-786a216dee39\") " pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs"
Jul 28 04:21:41.763927 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.763788    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b3ebb7d-856f-4e16-9e04-786a216dee39-cert\") pod \"odh-model-controller-7998c67dc4-pgzfs\" (UID: \"4b3ebb7d-856f-4e16-9e04-786a216dee39\") " pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs"
Jul 28 04:21:41.763988 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:21:41.763935    2574 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 04:21:41.764046 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:21:41.763991    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3ebb7d-856f-4e16-9e04-786a216dee39-cert podName:4b3ebb7d-856f-4e16-9e04-786a216dee39 nodeName:}" failed. No retries permitted until 2026-07-28 04:21:42.263976102 +0000 UTC m=+597.205815803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4b3ebb7d-856f-4e16-9e04-786a216dee39-cert") pod "odh-model-controller-7998c67dc4-pgzfs" (UID: "4b3ebb7d-856f-4e16-9e04-786a216dee39") : secret "odh-model-controller-webhook-cert" not found
Jul 28 04:21:41.773712 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.773688    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf8kj\" (UniqueName: \"kubernetes.io/projected/4b3ebb7d-856f-4e16-9e04-786a216dee39-kube-api-access-qf8kj\") pod \"odh-model-controller-7998c67dc4-pgzfs\" (UID: \"4b3ebb7d-856f-4e16-9e04-786a216dee39\") " pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs"
Jul 28 04:21:41.820047 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.820022    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:41.920125 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:41.920042    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:42.020563 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.020530    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:42.134973 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.134939    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:42.182559 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.182483    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5zmc7"]
Jul 28 04:21:42.185047 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.185025    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5zmc7"
Jul 28 04:21:42.187512 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.187489    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 28 04:21:42.187621 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.187527    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-ngptq\""
Jul 28 04:21:42.197774 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.197756    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5zmc7"]
Jul 28 04:21:42.268699 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.268671    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4df97ea0-3f6e-467c-b450-6d795d40119d-tls-certs\") pod \"model-serving-api-54d89bd5fc-5zmc7\" (UID: \"4df97ea0-3f6e-467c-b450-6d795d40119d\") " pod="opendatahub/model-serving-api-54d89bd5fc-5zmc7"
Jul 28 04:21:42.268854 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.268738    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgfdz\" (UniqueName: \"kubernetes.io/projected/4df97ea0-3f6e-467c-b450-6d795d40119d-kube-api-access-rgfdz\") pod \"model-serving-api-54d89bd5fc-5zmc7\" (UID: \"4df97ea0-3f6e-467c-b450-6d795d40119d\") " pod="opendatahub/model-serving-api-54d89bd5fc-5zmc7"
Jul 28 04:21:42.268854 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.268787    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b3ebb7d-856f-4e16-9e04-786a216dee39-cert\") pod \"odh-model-controller-7998c67dc4-pgzfs\" (UID: \"4b3ebb7d-856f-4e16-9e04-786a216dee39\") " pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs"
Jul 28 04:21:42.271192 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.271167    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b3ebb7d-856f-4e16-9e04-786a216dee39-cert\") pod \"odh-model-controller-7998c67dc4-pgzfs\" (UID: \"4b3ebb7d-856f-4e16-9e04-786a216dee39\") " pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs"
Jul 28 04:21:42.309499 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.309474    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:42.369986 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.369953    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4df97ea0-3f6e-467c-b450-6d795d40119d-tls-certs\") pod \"model-serving-api-54d89bd5fc-5zmc7\" (UID: \"4df97ea0-3f6e-467c-b450-6d795d40119d\") " pod="opendatahub/model-serving-api-54d89bd5fc-5zmc7"
Jul 28 04:21:42.370138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.370008    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgfdz\" (UniqueName: \"kubernetes.io/projected/4df97ea0-3f6e-467c-b450-6d795d40119d-kube-api-access-rgfdz\") pod \"model-serving-api-54d89bd5fc-5zmc7\" (UID: \"4df97ea0-3f6e-467c-b450-6d795d40119d\") " pod="opendatahub/model-serving-api-54d89bd5fc-5zmc7"
Jul 28 04:21:42.372631 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.372600    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4df97ea0-3f6e-467c-b450-6d795d40119d-tls-certs\") pod \"model-serving-api-54d89bd5fc-5zmc7\" (UID: \"4df97ea0-3f6e-467c-b450-6d795d40119d\") " pod="opendatahub/model-serving-api-54d89bd5fc-5zmc7"
Jul 28 04:21:42.377041 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.377014    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgfdz\" (UniqueName: \"kubernetes.io/projected/4df97ea0-3f6e-467c-b450-6d795d40119d-kube-api-access-rgfdz\") pod \"model-serving-api-54d89bd5fc-5zmc7\" (UID: \"4df97ea0-3f6e-467c-b450-6d795d40119d\") " pod="opendatahub/model-serving-api-54d89bd5fc-5zmc7"
Jul 28 04:21:42.495781 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.495692    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5zmc7"
Jul 28 04:21:42.505809 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.505789    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs"
Jul 28 04:21:42.641572 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.641547    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5zmc7"]
Jul 28 04:21:42.644901 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:21:42.644876    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df97ea0_3f6e_467c_b450_6d795d40119d.slice/crio-119351e60d0b61e378f2461778e5161875be707d24bbf851a635f7776de0944e WatchSource:0}: Error finding container 119351e60d0b61e378f2461778e5161875be707d24bbf851a635f7776de0944e: Status 404 returned error can't find the container with id 119351e60d0b61e378f2461778e5161875be707d24bbf851a635f7776de0944e
Jul 28 04:21:42.659609 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.659587    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pgzfs"]
Jul 28 04:21:42.661845 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:21:42.661812    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b3ebb7d_856f_4e16_9e04_786a216dee39.slice/crio-39e27bc69088a53382641841c60d9a617f780b8e28c47429fb9bce471f2fcae8 WatchSource:0}: Error finding container 39e27bc69088a53382641841c60d9a617f780b8e28c47429fb9bce471f2fcae8: Status 404 returned error can't find the container with id 39e27bc69088a53382641841c60d9a617f780b8e28c47429fb9bce471f2fcae8
Jul 28 04:21:42.667213 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:42.667185    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5zmc7" event={"ID":"4df97ea0-3f6e-467c-b450-6d795d40119d","Type":"ContainerStarted","Data":"119351e60d0b61e378f2461778e5161875be707d24bbf851a635f7776de0944e"}
Jul 28 04:21:43.674909 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:43.674868    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs" event={"ID":"4b3ebb7d-856f-4e16-9e04-786a216dee39","Type":"ContainerStarted","Data":"39e27bc69088a53382641841c60d9a617f780b8e28c47429fb9bce471f2fcae8"}
Jul 28 04:21:44.682825 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:44.682794    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5zmc7" event={"ID":"4df97ea0-3f6e-467c-b450-6d795d40119d","Type":"ContainerStarted","Data":"73ede27710ed22fd56bbff486054988a5d5b04431069b1fd1c13e8ac86c718b2"}
Jul 28 04:21:44.683307 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:44.683291    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-5zmc7"
Jul 28 04:21:44.698812 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:44.698768    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-5zmc7" podStartSLOduration=0.741674197 podStartE2EDuration="2.698750813s" totalImagesPullingTime="1.957076616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:21:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:21:44.697048863 +0000 UTC m=+599.638888583" watchObservedRunningTime="2026-07-28 04:21:44.698750813 +0000 UTC m=+599.640590579"
Jul 28 04:21:46.692949 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:46.692814    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs" event={"ID":"4b3ebb7d-856f-4e16-9e04-786a216dee39","Type":"ContainerStarted","Data":"57ebacbbf788db74a735bd080da5ba67645aaf05a9c8a3bc73beaed8c2e24ed8"}
Jul 28 04:21:46.707793 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:46.693026    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs"
Jul 28 04:21:46.707793 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:46.707033    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs" podStartSLOduration=2.291958369 podStartE2EDuration="5.707021852s" totalImagesPullingTime="3.415063483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:21:46.705864591 +0000 UTC m=+601.647704291" watchObservedRunningTime="2026-07-28 04:21:46.707021852 +0000 UTC m=+601.648861572"
Jul 28 04:21:51.256801 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.256771    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:51.333032 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.332995    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk"]
Jul 28 04:21:51.335617 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.335593    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk"
Jul 28 04:21:51.337755 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.337733    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 28 04:21:51.337875 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.337753    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-42mj7\""
Jul 28 04:21:51.345549 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.345524    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk"]
Jul 28 04:21:51.350907 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.350887    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"]
Jul 28 04:21:51.353089 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.353074    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"
Jul 28 04:21:51.355097 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.355080    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-f8rwv\""
Jul 28 04:21:51.355196 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.355142    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 28 04:21:51.363525 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.363504    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"]
Jul 28 04:21:51.449190 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.449159    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2hw8\" (UniqueName: \"kubernetes.io/projected/01bd8eb2-f8d0-4b6a-9709-36b19d94bf85-kube-api-access-x2hw8\") pod \"llmisvc-controller-manager-777c6bd7d5-zln4n\" (UID: \"01bd8eb2-f8d0-4b6a-9709-36b19d94bf85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"
Jul 28 04:21:51.449321 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.449207    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxzsl\" (UniqueName: \"kubernetes.io/projected/9407f103-7ffb-4eb6-8bf2-7abd5c801b80-kube-api-access-bxzsl\") pod \"kserve-controller-manager-d6f4b65b8-vkgnk\" (UID: \"9407f103-7ffb-4eb6-8bf2-7abd5c801b80\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk"
Jul 28 04:21:51.449321 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.449243    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01bd8eb2-f8d0-4b6a-9709-36b19d94bf85-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zln4n\" (UID: \"01bd8eb2-f8d0-4b6a-9709-36b19d94bf85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"
Jul 28 04:21:51.449321 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.449280    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9407f103-7ffb-4eb6-8bf2-7abd5c801b80-cert\") pod \"kserve-controller-manager-d6f4b65b8-vkgnk\" (UID: \"9407f103-7ffb-4eb6-8bf2-7abd5c801b80\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk"
Jul 28 04:21:51.550155 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.550127    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxzsl\" (UniqueName: \"kubernetes.io/projected/9407f103-7ffb-4eb6-8bf2-7abd5c801b80-kube-api-access-bxzsl\") pod \"kserve-controller-manager-d6f4b65b8-vkgnk\" (UID: \"9407f103-7ffb-4eb6-8bf2-7abd5c801b80\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk"
Jul 28 04:21:51.550287 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.550162    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01bd8eb2-f8d0-4b6a-9709-36b19d94bf85-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zln4n\" (UID: \"01bd8eb2-f8d0-4b6a-9709-36b19d94bf85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"
Jul 28 04:21:51.550287 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.550194    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9407f103-7ffb-4eb6-8bf2-7abd5c801b80-cert\") pod \"kserve-controller-manager-d6f4b65b8-vkgnk\" (UID: \"9407f103-7ffb-4eb6-8bf2-7abd5c801b80\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk"
Jul 28 04:21:51.550287 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.550225    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2hw8\" (UniqueName: \"kubernetes.io/projected/01bd8eb2-f8d0-4b6a-9709-36b19d94bf85-kube-api-access-x2hw8\") pod \"llmisvc-controller-manager-777c6bd7d5-zln4n\" (UID: \"01bd8eb2-f8d0-4b6a-9709-36b19d94bf85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"
Jul 28 04:21:51.550287 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:21:51.550277    2574 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 28 04:21:51.550530 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:21:51.550338    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01bd8eb2-f8d0-4b6a-9709-36b19d94bf85-cert podName:01bd8eb2-f8d0-4b6a-9709-36b19d94bf85 nodeName:}" failed. No retries permitted until 2026-07-28 04:21:52.050319264 +0000 UTC m=+606.992158972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/01bd8eb2-f8d0-4b6a-9709-36b19d94bf85-cert") pod "llmisvc-controller-manager-777c6bd7d5-zln4n" (UID: "01bd8eb2-f8d0-4b6a-9709-36b19d94bf85") : secret "llmisvc-webhook-server-cert" not found
Jul 28 04:21:51.552786 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.552763    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9407f103-7ffb-4eb6-8bf2-7abd5c801b80-cert\") pod \"kserve-controller-manager-d6f4b65b8-vkgnk\" (UID: \"9407f103-7ffb-4eb6-8bf2-7abd5c801b80\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk"
Jul 28 04:21:51.582496 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.582471    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxzsl\" (UniqueName: \"kubernetes.io/projected/9407f103-7ffb-4eb6-8bf2-7abd5c801b80-kube-api-access-bxzsl\") pod \"kserve-controller-manager-d6f4b65b8-vkgnk\" (UID: \"9407f103-7ffb-4eb6-8bf2-7abd5c801b80\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk"
Jul 28 04:21:51.582778 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.582760    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2hw8\" (UniqueName: \"kubernetes.io/projected/01bd8eb2-f8d0-4b6a-9709-36b19d94bf85-kube-api-access-x2hw8\") pod \"llmisvc-controller-manager-777c6bd7d5-zln4n\" (UID: \"01bd8eb2-f8d0-4b6a-9709-36b19d94bf85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"
Jul 28 04:21:51.645963 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.645940    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk"
Jul 28 04:21:51.772296 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.772254    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk"]
Jul 28 04:21:51.774380 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:21:51.774337    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9407f103_7ffb_4eb6_8bf2_7abd5c801b80.slice/crio-15ebfc42a9b8e169a1d2d63aa74d7badb01c4ac2c97d9cbbea4d251a0b54b51e WatchSource:0}: Error finding container 15ebfc42a9b8e169a1d2d63aa74d7badb01c4ac2c97d9cbbea4d251a0b54b51e: Status 404 returned error can't find the container with id 15ebfc42a9b8e169a1d2d63aa74d7badb01c4ac2c97d9cbbea4d251a0b54b51e
Jul 28 04:21:51.969825 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.969744    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-58d69b594d-z46js"]
Jul 28 04:21:51.973240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.973217    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js"
Jul 28 04:21:51.975598 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.975572    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 04:21:51.975739 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.975581    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-c94xc\""
Jul 28 04:21:51.975739 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.975579    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 04:21:51.975739 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.975613    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 04:21:51.975893 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.975784    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 04:21:51.981701 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:51.981484    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58d69b594d-z46js"]
Jul 28 04:21:52.055052 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.055019    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebe57051-0cab-4985-82eb-a0f619d1b35f-tls-certs\") pod \"kube-auth-proxy-58d69b594d-z46js\" (UID: \"ebe57051-0cab-4985-82eb-a0f619d1b35f\") " pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js"
Jul 28 04:21:52.055277 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.055075    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01bd8eb2-f8d0-4b6a-9709-36b19d94bf85-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zln4n\" (UID: \"01bd8eb2-f8d0-4b6a-9709-36b19d94bf85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"
Jul 28 04:21:52.055277 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.055098    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmd56\" (UniqueName: \"kubernetes.io/projected/ebe57051-0cab-4985-82eb-a0f619d1b35f-kube-api-access-gmd56\") pod \"kube-auth-proxy-58d69b594d-z46js\" (UID: \"ebe57051-0cab-4985-82eb-a0f619d1b35f\") " pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js"
Jul 28 04:21:52.055277 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.055139    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebe57051-0cab-4985-82eb-a0f619d1b35f-tmp\") pod \"kube-auth-proxy-58d69b594d-z46js\" (UID: \"ebe57051-0cab-4985-82eb-a0f619d1b35f\") " pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js"
Jul 28 04:21:52.057421 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.057392    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01bd8eb2-f8d0-4b6a-9709-36b19d94bf85-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zln4n\" (UID: \"01bd8eb2-f8d0-4b6a-9709-36b19d94bf85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"
Jul 28 04:21:52.156131 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.156101    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmd56\" (UniqueName: \"kubernetes.io/projected/ebe57051-0cab-4985-82eb-a0f619d1b35f-kube-api-access-gmd56\") pod \"kube-auth-proxy-58d69b594d-z46js\" (UID: \"ebe57051-0cab-4985-82eb-a0f619d1b35f\") " pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js"
Jul 28 04:21:52.156226 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.156146    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebe57051-0cab-4985-82eb-a0f619d1b35f-tmp\") pod \"kube-auth-proxy-58d69b594d-z46js\" (UID: \"ebe57051-0cab-4985-82eb-a0f619d1b35f\") " pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js"
Jul 28 04:21:52.156341 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.156323    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebe57051-0cab-4985-82eb-a0f619d1b35f-tls-certs\") pod \"kube-auth-proxy-58d69b594d-z46js\" (UID: \"ebe57051-0cab-4985-82eb-a0f619d1b35f\") " pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js"
Jul 28 04:21:52.156483 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:21:52.156465    2574 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 28 04:21:52.156549 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:21:52.156539    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe57051-0cab-4985-82eb-a0f619d1b35f-tls-certs podName:ebe57051-0cab-4985-82eb-a0f619d1b35f nodeName:}" failed. No retries permitted until 2026-07-28 04:21:52.656523338 +0000 UTC m=+607.598363040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ebe57051-0cab-4985-82eb-a0f619d1b35f-tls-certs") pod "kube-auth-proxy-58d69b594d-z46js" (UID: "ebe57051-0cab-4985-82eb-a0f619d1b35f") : secret "kube-auth-proxy-tls" not found
Jul 28 04:21:52.158214 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.158178    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebe57051-0cab-4985-82eb-a0f619d1b35f-tmp\") pod \"kube-auth-proxy-58d69b594d-z46js\" (UID: \"ebe57051-0cab-4985-82eb-a0f619d1b35f\") " pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js"
Jul 28 04:21:52.168879 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.168857    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmd56\" (UniqueName: \"kubernetes.io/projected/ebe57051-0cab-4985-82eb-a0f619d1b35f-kube-api-access-gmd56\") pod \"kube-auth-proxy-58d69b594d-z46js\" (UID: \"ebe57051-0cab-4985-82eb-a0f619d1b35f\") " pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js"
Jul 28 04:21:52.262642 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.262595    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"
Jul 28 04:21:52.384795 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.384768    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"]
Jul 28 04:21:52.386860 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:21:52.386834    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod01bd8eb2_f8d0_4b6a_9709_36b19d94bf85.slice/crio-aa328fac8af9e38060032de48c4af52790ef10ff60680d52ae3edee89acb4a93 WatchSource:0}: Error finding container aa328fac8af9e38060032de48c4af52790ef10ff60680d52ae3edee89acb4a93: Status 404 returned error can't find the container with id aa328fac8af9e38060032de48c4af52790ef10ff60680d52ae3edee89acb4a93
Jul 28 04:21:52.659964 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.659936    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebe57051-0cab-4985-82eb-a0f619d1b35f-tls-certs\") pod \"kube-auth-proxy-58d69b594d-z46js\" (UID: \"ebe57051-0cab-4985-82eb-a0f619d1b35f\") " pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js"
Jul 28 04:21:52.662582 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.662559    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebe57051-0cab-4985-82eb-a0f619d1b35f-tls-certs\") pod \"kube-auth-proxy-58d69b594d-z46js\" (UID: \"ebe57051-0cab-4985-82eb-a0f619d1b35f\") " pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js"
Jul 28 04:21:52.706827 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.706793    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:52.719327 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.719297    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n" event={"ID":"01bd8eb2-f8d0-4b6a-9709-36b19d94bf85","Type":"ContainerStarted","Data":"aa328fac8af9e38060032de48c4af52790ef10ff60680d52ae3edee89acb4a93"}
Jul 28 04:21:52.720589 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.720565    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk" event={"ID":"9407f103-7ffb-4eb6-8bf2-7abd5c801b80","Type":"ContainerStarted","Data":"15ebfc42a9b8e169a1d2d63aa74d7badb01c4ac2c97d9cbbea4d251a0b54b51e"}
Jul 28 04:21:52.885481 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:52.885404    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js"
Jul 28 04:21:53.047403 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.047377    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58d69b594d-z46js"]
Jul 28 04:21:53.072153 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:21:53.072122    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe57051_0cab_4985_82eb_a0f619d1b35f.slice/crio-98f8c657ecd3b2216004a7895471f7153626ca399674f74e2acb46794570f70d WatchSource:0}: Error finding container 98f8c657ecd3b2216004a7895471f7153626ca399674f74e2acb46794570f70d: Status 404 returned error can't find the container with id 98f8c657ecd3b2216004a7895471f7153626ca399674f74e2acb46794570f70d
Jul 28 04:21:53.653718 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.653685    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:53.661675 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.661641    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"]
Jul 28 04:21:53.664849 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.664819    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.668010 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.667992    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 04:21:53.668408 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.668386    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kz4tw\""
Jul 28 04:21:53.668513 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.668500    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 04:21:53.668577 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.668503    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 04:21:53.668760 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.668741    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 04:21:53.682924 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.682903    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"]
Jul 28 04:21:53.727100 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.727076    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js" event={"ID":"ebe57051-0cab-4985-82eb-a0f619d1b35f","Type":"ContainerStarted","Data":"98f8c657ecd3b2216004a7895471f7153626ca399674f74e2acb46794570f70d"}
Jul 28 04:21:53.747923 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.747894    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:53.770006 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.769979    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c446cc-aab6-4231-b1bf-eb763e839406-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.770144 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.770012    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1c446cc-aab6-4231-b1bf-eb763e839406-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.770144 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.770039    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n4ch\" (UniqueName: \"kubernetes.io/projected/d1c446cc-aab6-4231-b1bf-eb763e839406-kube-api-access-4n4ch\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.770144 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.770134    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1c446cc-aab6-4231-b1bf-eb763e839406-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.770312 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.770213    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1c446cc-aab6-4231-b1bf-eb763e839406-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.770312 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.770252    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1c446cc-aab6-4231-b1bf-eb763e839406-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.770430 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.770339    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c446cc-aab6-4231-b1bf-eb763e839406-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.871804 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.871715    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1c446cc-aab6-4231-b1bf-eb763e839406-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.871804 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.871767    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1c446cc-aab6-4231-b1bf-eb763e839406-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.871804 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.871806    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c446cc-aab6-4231-b1bf-eb763e839406-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.872076 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.871858    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c446cc-aab6-4231-b1bf-eb763e839406-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.872076 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.871882    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1c446cc-aab6-4231-b1bf-eb763e839406-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.872076 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.871904    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n4ch\" (UniqueName: \"kubernetes.io/projected/d1c446cc-aab6-4231-b1bf-eb763e839406-kube-api-access-4n4ch\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.872076 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.871937    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1c446cc-aab6-4231-b1bf-eb763e839406-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.872477 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.872450    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1c446cc-aab6-4231-b1bf-eb763e839406-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.875480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.875458    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c446cc-aab6-4231-b1bf-eb763e839406-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.876195 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.876170    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1c446cc-aab6-4231-b1bf-eb763e839406-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.876636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.876620    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1c446cc-aab6-4231-b1bf-eb763e839406-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.876957 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.876913    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1c446cc-aab6-4231-b1bf-eb763e839406-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.880984 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.880958    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c446cc-aab6-4231-b1bf-eb763e839406-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.881583 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.881549    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n4ch\" (UniqueName: \"kubernetes.io/projected/d1c446cc-aab6-4231-b1bf-eb763e839406-kube-api-access-4n4ch\") pod \"istiod-openshift-gateway-5677ccb49b-zr6b6\" (UID: \"d1c446cc-aab6-4231-b1bf-eb763e839406\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:53.977266 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:53.977197    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:54.408378 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:54.408336    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"]
Jul 28 04:21:54.410848 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:21:54.410817    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c446cc_aab6_4231_b1bf_eb763e839406.slice/crio-8631c9ff8bbb54b624508965be24a4422f4d2e8b704b3c928acb4746f14f33bf WatchSource:0}: Error finding container 8631c9ff8bbb54b624508965be24a4422f4d2e8b704b3c928acb4746f14f33bf: Status 404 returned error can't find the container with id 8631c9ff8bbb54b624508965be24a4422f4d2e8b704b3c928acb4746f14f33bf
Jul 28 04:21:54.734288 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:54.734198    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk" event={"ID":"9407f103-7ffb-4eb6-8bf2-7abd5c801b80","Type":"ContainerStarted","Data":"3bd30f24710548214861832414f993506fdfea64f29d276185518eefe730514d"}
Jul 28 04:21:54.734773 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:54.734470    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk"
Jul 28 04:21:54.735770 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:54.735742    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6" event={"ID":"d1c446cc-aab6-4231-b1bf-eb763e839406","Type":"ContainerStarted","Data":"8631c9ff8bbb54b624508965be24a4422f4d2e8b704b3c928acb4746f14f33bf"}
Jul 28 04:21:54.875900 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:54.875864    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:56.699629 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:56.699600    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-5zmc7"
Jul 28 04:21:56.736034 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:56.735976    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk" podStartSLOduration=3.230113221 podStartE2EDuration="5.735955925s" totalImagesPullingTime="2.505842704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:21:54.766523611 +0000 UTC m=+609.708363332" watchObservedRunningTime="2026-07-28 04:21:56.735955925 +0000 UTC m=+611.677795645"
Jul 28 04:21:57.558067 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:57.558028    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 04:21:57.558163 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:57.558117    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 04:21:57.700312 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:57.700284    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-pgzfs"
Jul 28 04:21:57.752462 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:57.752423    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js" event={"ID":"ebe57051-0cab-4985-82eb-a0f619d1b35f","Type":"ContainerStarted","Data":"1a8cf0dc31d7d609440d05b8326408f3381312b6c6b56dbca0d908eb7ff1ac44"}
Jul 28 04:21:57.754202 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:57.754171    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6" event={"ID":"d1c446cc-aab6-4231-b1bf-eb763e839406","Type":"ContainerStarted","Data":"d2e874d1d160af4e5c70d4ee77fca1c46fadff809a70ba9e74701a8f74b9f37f"}
Jul 28 04:21:57.754345 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:57.754227    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:21:57.756283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:57.756256    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n" event={"ID":"01bd8eb2-f8d0-4b6a-9709-36b19d94bf85","Type":"ContainerStarted","Data":"8cb055203bb65d17e476e82238b888521baccc94185ba3db768a46ab4f485af1"}
Jul 28 04:21:57.756440 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:57.756311    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"
Jul 28 04:21:57.767125 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:57.767041    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-58d69b594d-z46js" podStartSLOduration=2.290067304 podStartE2EDuration="6.767024639s" totalImagesPullingTime="4.476957335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:21:57.765859791 +0000 UTC m=+612.707699512" watchObservedRunningTime="2026-07-28 04:21:57.767024639 +0000 UTC m=+612.708864358"
Jul 28 04:21:57.783797 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:57.783758    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6" podStartSLOduration=1.6390413860000002 podStartE2EDuration="4.783745792s" totalImagesPullingTime="3.144704406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:21:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:21:57.78184 +0000 UTC m=+612.723679721" watchObservedRunningTime="2026-07-28 04:21:57.783745792 +0000 UTC m=+612.725585513"
Jul 28 04:21:57.797404 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:57.797349    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n" podStartSLOduration=1.6318174399999998 podStartE2EDuration="6.797337696s" totalImagesPullingTime="5.165520256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:21:57.795300637 +0000 UTC m=+612.737140360" watchObservedRunningTime="2026-07-28 04:21:57.797337696 +0000 UTC m=+612.739177416"
Jul 28 04:21:58.763994 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:58.763958    2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-zr6b6 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 04:21:58.764459 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:58.764028    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6" podUID="d1c446cc-aab6-4231-b1bf-eb763e839406" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 04:21:59.133270 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:59.133229    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:59.233438 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:59.233401    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:59.820533 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:59.820505    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:21:59.980572 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:21:59.980537    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:22:00.077600 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:22:00.077519    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:22:01.763314 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:22:01.763290    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6"
Jul 28 04:22:23.506252 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:22:23.506164    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:22:25.746477 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:22:25.746443    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vkgnk"
Jul 28 04:22:28.764223 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:22:28.764194    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zln4n"
Jul 28 04:22:42.453753 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:22:42.453716    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:22:42.553707 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:22:42.553675    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:22:42.634561 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:22:42.634521    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:11.157446 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:11.157407    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:15.388127 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:15.388091    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:16.395398 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:16.395344    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:16.761605 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:16.761532    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:18.214925 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:18.214894    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:18.322527 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:18.322493    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:19.561516 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:19.561476    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:19.776098 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:19.776057    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:19.959700 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:19.959629    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:20.152862 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:20.152831    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:20.254318 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:20.254249    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:20.677593 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:20.677561    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:21.963910 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:21.963881    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:22.065592 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:22.065564    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:25.320137 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:25.320106    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:25.376897 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:25.376871    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:26.154795 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:26.154767    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:26.203254 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:26.203231    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:26.693015 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:26.692989    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:31.006632 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:31.006602    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:31.107187 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:31.107149    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:31.207425 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:31.207393    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:34.822125 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.822090    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"]
Jul 28 04:23:34.824532 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.824516    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"
Jul 28 04:23:34.827107 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.827083    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-s2lcg\""
Jul 28 04:23:34.828018 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.828003    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 04:23:34.828290 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.828275    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 04:23:34.844636 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.844612    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"]
Jul 28 04:23:34.873241 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.873222    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v4rg\" (UniqueName: \"kubernetes.io/projected/9c93ae82-e2e3-457b-8f6d-de16e6102ef9-kube-api-access-7v4rg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7c69c\" (UID: \"9c93ae82-e2e3-457b-8f6d-de16e6102ef9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"
Jul 28 04:23:34.873338 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.873256    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c93ae82-e2e3-457b-8f6d-de16e6102ef9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7c69c\" (UID: \"9c93ae82-e2e3-457b-8f6d-de16e6102ef9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"
Jul 28 04:23:34.974465 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.974434    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4rg\" (UniqueName: \"kubernetes.io/projected/9c93ae82-e2e3-457b-8f6d-de16e6102ef9-kube-api-access-7v4rg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7c69c\" (UID: \"9c93ae82-e2e3-457b-8f6d-de16e6102ef9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"
Jul 28 04:23:34.974613 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.974481    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c93ae82-e2e3-457b-8f6d-de16e6102ef9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7c69c\" (UID: \"9c93ae82-e2e3-457b-8f6d-de16e6102ef9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"
Jul 28 04:23:34.974885 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.974858    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c93ae82-e2e3-457b-8f6d-de16e6102ef9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7c69c\" (UID: \"9c93ae82-e2e3-457b-8f6d-de16e6102ef9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"
Jul 28 04:23:34.991208 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.991182    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v4rg\" (UniqueName: \"kubernetes.io/projected/9c93ae82-e2e3-457b-8f6d-de16e6102ef9-kube-api-access-7v4rg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7c69c\" (UID: \"9c93ae82-e2e3-457b-8f6d-de16e6102ef9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"
Jul 28 04:23:34.997111 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.997089    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"]
Jul 28 04:23:34.997336 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:34.997321    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"
Jul 28 04:23:35.011517 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.011492    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:35.111747 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.111713    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:35.144149 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.144125    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"]
Jul 28 04:23:35.146671 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:23:35.146645    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c93ae82_e2e3_457b_8f6d_de16e6102ef9.slice/crio-e37b8524828d3e46ddaa220e587925fffd9dd11e2fc0c00f283a97827795b15a WatchSource:0}: Error finding container e37b8524828d3e46ddaa220e587925fffd9dd11e2fc0c00f283a97827795b15a: Status 404 returned error can't find the container with id e37b8524828d3e46ddaa220e587925fffd9dd11e2fc0c00f283a97827795b15a
Jul 28 04:23:35.149096 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.149082    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 04:23:35.184381 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.184327    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c" event={"ID":"9c93ae82-e2e3-457b-8f6d-de16e6102ef9","Type":"ContainerStarted","Data":"e37b8524828d3e46ddaa220e587925fffd9dd11e2fc0c00f283a97827795b15a"}
Jul 28 04:23:35.378606 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.378543    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"]
Jul 28 04:23:35.387963 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.387941    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"]
Jul 28 04:23:35.453694 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.453667    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lps4s"]
Jul 28 04:23:35.456740 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.456723    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lps4s"
Jul 28 04:23:35.458947 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.458930    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-xt2pg\""
Jul 28 04:23:35.467275 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.467253    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lps4s"]
Jul 28 04:23:35.583587 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.583557    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hplfw\" (UniqueName: \"kubernetes.io/projected/628810bb-4c2f-44d7-8be0-42d1a8fdb8ef-kube-api-access-hplfw\") pod \"limitador-operator-controller-manager-859d7b78bd-lps4s\" (UID: \"628810bb-4c2f-44d7-8be0-42d1a8fdb8ef\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lps4s"
Jul 28 04:23:35.684648 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.684580    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hplfw\" (UniqueName: \"kubernetes.io/projected/628810bb-4c2f-44d7-8be0-42d1a8fdb8ef-kube-api-access-hplfw\") pod \"limitador-operator-controller-manager-859d7b78bd-lps4s\" (UID: \"628810bb-4c2f-44d7-8be0-42d1a8fdb8ef\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lps4s"
Jul 28 04:23:35.697756 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.697734    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hplfw\" (UniqueName: \"kubernetes.io/projected/628810bb-4c2f-44d7-8be0-42d1a8fdb8ef-kube-api-access-hplfw\") pod \"limitador-operator-controller-manager-859d7b78bd-lps4s\" (UID: \"628810bb-4c2f-44d7-8be0-42d1a8fdb8ef\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lps4s"
Jul 28 04:23:35.768170 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:35.768130    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lps4s"
Jul 28 04:23:36.238953 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:36.238880    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lps4s"]
Jul 28 04:23:36.296811 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:23:36.296767    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod628810bb_4c2f_44d7_8be0_42d1a8fdb8ef.slice/crio-f51f2b019664b83c2b1c46aaf401f26c32a74161fcfc0e02e2de99f45d8235e7 WatchSource:0}: Error finding container f51f2b019664b83c2b1c46aaf401f26c32a74161fcfc0e02e2de99f45d8235e7: Status 404 returned error can't find the container with id f51f2b019664b83c2b1c46aaf401f26c32a74161fcfc0e02e2de99f45d8235e7
Jul 28 04:23:37.197134 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:37.197101    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lps4s" event={"ID":"628810bb-4c2f-44d7-8be0-42d1a8fdb8ef","Type":"ContainerStarted","Data":"f51f2b019664b83c2b1c46aaf401f26c32a74161fcfc0e02e2de99f45d8235e7"}
Jul 28 04:23:37.422346 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:37.422265    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:37.513446 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:37.513339    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:38.202779 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:38.202748    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lps4s" event={"ID":"628810bb-4c2f-44d7-8be0-42d1a8fdb8ef","Type":"ContainerStarted","Data":"ed3406fe9fb07bbed64e608e16b76682bbb13da06b89ee0f57612148ae738073"}
Jul 28 04:23:38.203005 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:38.202989    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lps4s"
Jul 28 04:23:38.218490 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:38.218444    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lps4s" podStartSLOduration=1.825225716 podStartE2EDuration="3.218431687s" totalImagesPullingTime="1.393205971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:23:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:23:38.217108404 +0000 UTC m=+713.158948126" watchObservedRunningTime="2026-07-28 04:23:38.218431687 +0000 UTC m=+713.160271406"
Jul 28 04:23:40.215174 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:40.215143    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c" podUID="9c93ae82-e2e3-457b-8f6d-de16e6102ef9" containerName="manager" containerID="cri-o://6453def715bbeef17ddd4f8b137a7e523ab784359b24a70510a6e4148c834492" gracePeriod=2
Jul 28 04:23:40.446923 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:40.446900    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"
Jul 28 04:23:40.448581 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:23:40.448559    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-7c69c\" is forbidden: User \"system:node:ip-10-0-133-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-204.ec2.internal' and this object" podUID="9c93ae82-e2e3-457b-8f6d-de16e6102ef9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"
Jul 28 04:23:40.528121 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:40.528104    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c93ae82-e2e3-457b-8f6d-de16e6102ef9-extensions-socket-volume\") pod \"9c93ae82-e2e3-457b-8f6d-de16e6102ef9\" (UID: \"9c93ae82-e2e3-457b-8f6d-de16e6102ef9\") "
Jul 28 04:23:40.528242 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:40.528137    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7v4rg\" (UniqueName: \"kubernetes.io/projected/9c93ae82-e2e3-457b-8f6d-de16e6102ef9-kube-api-access-7v4rg\") pod \"9c93ae82-e2e3-457b-8f6d-de16e6102ef9\" (UID: \"9c93ae82-e2e3-457b-8f6d-de16e6102ef9\") "
Jul 28 04:23:40.528409 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:40.528388    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c93ae82-e2e3-457b-8f6d-de16e6102ef9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9c93ae82-e2e3-457b-8f6d-de16e6102ef9" (UID: "9c93ae82-e2e3-457b-8f6d-de16e6102ef9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 04:23:40.530174 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:40.530151    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c93ae82-e2e3-457b-8f6d-de16e6102ef9-kube-api-access-7v4rg" (OuterVolumeSpecName: "kube-api-access-7v4rg") pod "9c93ae82-e2e3-457b-8f6d-de16e6102ef9" (UID: "9c93ae82-e2e3-457b-8f6d-de16e6102ef9"). InnerVolumeSpecName "kube-api-access-7v4rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 04:23:40.629655 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:40.629634    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c93ae82-e2e3-457b-8f6d-de16e6102ef9-extensions-socket-volume\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:23:40.629655 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:40.629653    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v4rg\" (UniqueName: \"kubernetes.io/projected/9c93ae82-e2e3-457b-8f6d-de16e6102ef9-kube-api-access-7v4rg\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:23:41.221015 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:41.220984    2574 generic.go:356] "Generic (PLEG): container finished" podID="9c93ae82-e2e3-457b-8f6d-de16e6102ef9" containerID="6453def715bbeef17ddd4f8b137a7e523ab784359b24a70510a6e4148c834492" exitCode=2
Jul 28 04:23:41.221519 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:41.221028    2574 scope.go:117] "RemoveContainer" containerID="6453def715bbeef17ddd4f8b137a7e523ab784359b24a70510a6e4148c834492"
Jul 28 04:23:41.221519 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:41.221033    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"
Jul 28 04:23:41.222901 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:23:41.222878    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-7c69c\" is forbidden: User \"system:node:ip-10-0-133-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-204.ec2.internal' and this object" podUID="9c93ae82-e2e3-457b-8f6d-de16e6102ef9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"
Jul 28 04:23:41.230160 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:41.230142    2574 scope.go:117] "RemoveContainer" containerID="6453def715bbeef17ddd4f8b137a7e523ab784359b24a70510a6e4148c834492"
Jul 28 04:23:41.230429 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:23:41.230408    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6453def715bbeef17ddd4f8b137a7e523ab784359b24a70510a6e4148c834492\": container with ID starting with 6453def715bbeef17ddd4f8b137a7e523ab784359b24a70510a6e4148c834492 not found: ID does not exist" containerID="6453def715bbeef17ddd4f8b137a7e523ab784359b24a70510a6e4148c834492"
Jul 28 04:23:41.230483 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:41.230438    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6453def715bbeef17ddd4f8b137a7e523ab784359b24a70510a6e4148c834492"} err="failed to get container status \"6453def715bbeef17ddd4f8b137a7e523ab784359b24a70510a6e4148c834492\": rpc error: code = NotFound desc = could not find container \"6453def715bbeef17ddd4f8b137a7e523ab784359b24a70510a6e4148c834492\": container with ID starting with 6453def715bbeef17ddd4f8b137a7e523ab784359b24a70510a6e4148c834492 not found: ID does not exist"
Jul 28 04:23:41.230755 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:23:41.230732    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-7c69c\" is forbidden: User \"system:node:ip-10-0-133-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-204.ec2.internal' and this object" podUID="9c93ae82-e2e3-457b-8f6d-de16e6102ef9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7c69c"
Jul 28 04:23:41.620370 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:41.620337    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c93ae82-e2e3-457b-8f6d-de16e6102ef9" path="/var/lib/kubelet/pods/9c93ae82-e2e3-457b-8f6d-de16e6102ef9/volumes"
Jul 28 04:23:42.169904 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:42.169870    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:42.269806 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:42.269779    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:46.164349 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:46.164317    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:46.242026 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:46.241994    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:49.211377 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:49.211326    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lps4s"
Jul 28 04:23:56.363076 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:56.363043    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:56.430761 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:56.430730    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:56.557054 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:56.557027    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:56.623760 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:56.623696    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:58.151599 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:58.151563    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:23:58.217453 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:23:58.217427    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:02.572179 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:02.572148    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:02.672601 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:02.672573    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:08.398343 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:08.398308    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:09.523675 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:09.523640    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:23.871342 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:23.871305    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:23.970155 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:23.970123    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:24.022099 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.022069    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:24.166971 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.166891    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:24:24.167258 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.167245    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c93ae82-e2e3-457b-8f6d-de16e6102ef9" containerName="manager"
Jul 28 04:24:24.167319 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.167260    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c93ae82-e2e3-457b-8f6d-de16e6102ef9" containerName="manager"
Jul 28 04:24:24.167384 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.167339    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c93ae82-e2e3-457b-8f6d-de16e6102ef9" containerName="manager"
Jul 28 04:24:24.171307 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.171288    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"
Jul 28 04:24:24.173652 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.173633    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 04:24:24.173742 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.173726    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cbl8r\""
Jul 28 04:24:24.179127 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.179109    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:24:24.204736 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.204711    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:24:24.250561 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.250539    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0-config-file\") pod \"limitador-limitador-54d6d988d5-sp9fh\" (UID: \"c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"
Jul 28 04:24:24.250660 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.250645    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h66l2\" (UniqueName: \"kubernetes.io/projected/c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0-kube-api-access-h66l2\") pod \"limitador-limitador-54d6d988d5-sp9fh\" (UID: \"c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"
Jul 28 04:24:24.351437 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.351401    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h66l2\" (UniqueName: \"kubernetes.io/projected/c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0-kube-api-access-h66l2\") pod \"limitador-limitador-54d6d988d5-sp9fh\" (UID: \"c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"
Jul 28 04:24:24.351545 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.351459    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0-config-file\") pod \"limitador-limitador-54d6d988d5-sp9fh\" (UID: \"c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"
Jul 28 04:24:24.352118 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.352101    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0-config-file\") pod \"limitador-limitador-54d6d988d5-sp9fh\" (UID: \"c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"
Jul 28 04:24:24.358240 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.358220    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h66l2\" (UniqueName: \"kubernetes.io/projected/c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0-kube-api-access-h66l2\") pod \"limitador-limitador-54d6d988d5-sp9fh\" (UID: \"c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"
Jul 28 04:24:24.481696 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.481646    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"
Jul 28 04:24:24.608868 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.608721    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:24:24.611310 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:24:24.611281    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc808b49a_2a89_4218_a1d4_4a8ea1a5f4d0.slice/crio-0f24bbc1098818b202811f50e73d9f1423dc052f21f348ad7f80c7fc3630b40e WatchSource:0}: Error finding container 0f24bbc1098818b202811f50e73d9f1423dc052f21f348ad7f80c7fc3630b40e: Status 404 returned error can't find the container with id 0f24bbc1098818b202811f50e73d9f1423dc052f21f348ad7f80c7fc3630b40e
Jul 28 04:24:24.684029 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.684003    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:24.763283 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:24.763212    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:25.427057 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:25.426971    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sp9fh" event={"ID":"c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0","Type":"ContainerStarted","Data":"0f24bbc1098818b202811f50e73d9f1423dc052f21f348ad7f80c7fc3630b40e"}
Jul 28 04:24:27.439082 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:27.438998    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sp9fh" event={"ID":"c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0","Type":"ContainerStarted","Data":"b74a287de5ec200b6013e30e5b8d49a97f05ae1a9a692d611f1892ed21dcf348"}
Jul 28 04:24:27.439534 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:27.439093    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"
Jul 28 04:24:27.455268 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:27.455221    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-sp9fh" podStartSLOduration=0.99472531 podStartE2EDuration="3.455205802s" totalImagesPullingTime="2.460480492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:24:27.453150703 +0000 UTC m=+762.394990424" watchObservedRunningTime="2026-07-28 04:24:27.455205802 +0000 UTC m=+762.397045522"
Jul 28 04:24:30.996873 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:30.996829    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:38.444698 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:38.444671    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"
Jul 28 04:24:42.530821 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:42.530790    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:42.629113 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:42.629081    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:43.090695 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:43.090664    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:55.389628 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:55.389576    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:24:55.629315 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:24:55.629281    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:03.027464 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:03.027433    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:03.275152 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:03.275123    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:03.325943 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:03.325865    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:03.497037 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:03.497000    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:06.676086 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:06.676045    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:06.779232 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:06.779199    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:08.905408 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:08.905320    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:39.319113 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:39.319070    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:39.487583 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:39.487545    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:39.962807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:39.962777    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:40.222453 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:40.222341    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:40.419979 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:40.419948    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:40.513196 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:40.513112    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:40.588332 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:40.588294    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:25:49.729540 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:49.729507    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:52.391236 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:52.391197    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:25:52.530350 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:25:52.530320    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:26:09.715540 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:09.715509    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:26:10.295625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:10.295587    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:26:16.115706 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:16.115676    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:26:16.864593 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:16.864562    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:26:22.219721 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:22.219689    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:26:23.368459 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:23.368421    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:26:30.182309 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.182276    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"]
Jul 28 04:26:30.186098 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.186074    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.188479 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.188457    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 04:26:30.189084 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.189067    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 04:26:30.189162 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.189106    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 28 04:26:30.189162 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.189071    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gmrkq\""
Jul 28 04:26:30.194867 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.194847    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"]
Jul 28 04:26:30.349695 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.349665    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39734416-47e7-436e-a8da-fb8bd981d198-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.349880 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.349713    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39734416-47e7-436e-a8da-fb8bd981d198-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.349880 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.349746    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39734416-47e7-436e-a8da-fb8bd981d198-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.349880 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.349769    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39734416-47e7-436e-a8da-fb8bd981d198-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.349880 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.349808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39734416-47e7-436e-a8da-fb8bd981d198-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.349880 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.349829    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62h5v\" (UniqueName: \"kubernetes.io/projected/39734416-47e7-436e-a8da-fb8bd981d198-kube-api-access-62h5v\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.385478 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.385442    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:26:30.451179 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.451099    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39734416-47e7-436e-a8da-fb8bd981d198-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.451179 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.451142    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39734416-47e7-436e-a8da-fb8bd981d198-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.451179 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.451162    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39734416-47e7-436e-a8da-fb8bd981d198-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.451460 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.451183    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39734416-47e7-436e-a8da-fb8bd981d198-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.451460 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.451212    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62h5v\" (UniqueName: \"kubernetes.io/projected/39734416-47e7-436e-a8da-fb8bd981d198-kube-api-access-62h5v\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.451460 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.451273    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39734416-47e7-436e-a8da-fb8bd981d198-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.451663 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.451637    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39734416-47e7-436e-a8da-fb8bd981d198-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.451663 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.451653    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39734416-47e7-436e-a8da-fb8bd981d198-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.451761 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.451676    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39734416-47e7-436e-a8da-fb8bd981d198-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.453552 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.453525    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39734416-47e7-436e-a8da-fb8bd981d198-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.453786 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.453769    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39734416-47e7-436e-a8da-fb8bd981d198-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.458045 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.458026    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62h5v\" (UniqueName: \"kubernetes.io/projected/39734416-47e7-436e-a8da-fb8bd981d198-kube-api-access-62h5v\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx\" (UID: \"39734416-47e7-436e-a8da-fb8bd981d198\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.498035 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.498011    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:30.624896 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.624856    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"]
Jul 28 04:26:30.627118 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:26:30.627091    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39734416_47e7_436e_a8da_fb8bd981d198.slice/crio-2ed45ab4958da7593ed4331cea9e05ab2400fb9eac6083778e6c4502e760d7f2 WatchSource:0}: Error finding container 2ed45ab4958da7593ed4331cea9e05ab2400fb9eac6083778e6c4502e760d7f2: Status 404 returned error can't find the container with id 2ed45ab4958da7593ed4331cea9e05ab2400fb9eac6083778e6c4502e760d7f2
Jul 28 04:26:30.998252 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:30.998212    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx" event={"ID":"39734416-47e7-436e-a8da-fb8bd981d198","Type":"ContainerStarted","Data":"2ed45ab4958da7593ed4331cea9e05ab2400fb9eac6083778e6c4502e760d7f2"}
Jul 28 04:26:31.859893 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:31.859806    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:26:32.005424 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:32.005388    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx" event={"ID":"39734416-47e7-436e-a8da-fb8bd981d198","Type":"ContainerStarted","Data":"dbbad920be449e20076eb1d021b14c386b774d3b47d0a50b644ee220a05839a8"}
Jul 28 04:26:32.022389 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:32.022331    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx" podStartSLOduration=1.083658975 podStartE2EDuration="2.022317485s" totalImagesPullingTime="938.65851ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:26:32.019891775 +0000 UTC m=+886.961731494" watchObservedRunningTime="2026-07-28 04:26:32.022317485 +0000 UTC m=+886.964157205"
Jul 28 04:26:35.681462 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:35.681413    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:26:37.999829 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:37.999748    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:26:40.499192 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:40.499151    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:40.499625 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:40.499223    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:40.511468 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:40.511446    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:41.059083 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:41.059052    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx"
Jul 28 04:26:41.758630 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:41.758591    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:26:50.028655 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:50.028616    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:26:52.461992 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:26:52.461958    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:27:21.238458 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:27:21.238399    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:27:46.971083 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:27:46.971049    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:28:09.029451 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:28:09.029376    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:28:09.080102 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:28:09.080069    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:28:36.205019 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:28:36.204990    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:28:36.304575 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:28:36.304544    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:28:37.450850 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:28:37.450813    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:28:37.553968 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:28:37.553922    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:28:45.726215 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:28:45.726176    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:29:06.466153 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:29:06.466120    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:29:08.573757 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:29:08.573721    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:29:08.625784 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:29:08.625756    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:29:16.467508 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:29:16.467472    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:29:27.067204 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:29:27.067167    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:30:43.464640 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:30:43.464606    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:30:44.687266 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:30:44.687222    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:31:09.668147 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:09.668044    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:31:17.362483 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:17.362452    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:31:17.461760 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:17.461721    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:31:17.601583 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:17.601553    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:31:18.367570 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:18.367536    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:31:23.248346 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:23.248311    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:31:23.312888 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:23.312862    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:31:23.398822 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:23.398783    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:31:23.498981 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:23.498900    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:31:23.607503 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:23.607480    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:31:29.293858 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:29.293808    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:31:29.354379 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:29.354325    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:31:42.413273 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:42.413212    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:31:42.513448 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:42.513419    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:31:46.060171 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:46.060129    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:31:55.955197 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:31:55.955166    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:32:05.268979 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:32:05.268947    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:32:13.562787 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:32:13.562749    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:32:24.059640 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:32:24.059608    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:32:33.461939 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:32:33.461905    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:32:43.667415 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:32:43.667340    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:32:53.766075 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:32:53.766038    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:33:15.662733 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:33:15.662698    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:33:27.567113 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:33:27.567079    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:33:50.963398 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:33:50.963354    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:33:54.569529 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:33:54.569496    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:33:55.879899 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:33:55.879866    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:34:01.162747 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:34:01.162710    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:34:11.372057 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:34:11.371976    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:34:27.172344 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:34:27.172305    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:34:37.466718 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:34:37.466687    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:34:46.770835 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:34:46.770799    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:34:55.863859 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:34:55.863826    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:35:05.155720 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:35:05.155683    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:37:00.576060 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:37:00.576022    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:37:11.775889 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:37:11.775808    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:37:22.545491 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:37:22.545458    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:37:27.194999 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:37:27.194961    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:37:35.500795 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:37:35.500759    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:37:35.601316 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:37:35.601280    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:37:39.175002 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:37:39.174966    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:37:48.184423 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:37:48.184393    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:37:57.591195 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:37:57.586044    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:38:09.076462 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:38:09.076424    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:38:19.878755 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:38:19.878723    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:38:28.088815 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:38:28.088782    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:38:39.182265 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:38:39.182193    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:38:51.175937 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:38:51.175902    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:38:58.977497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:38:58.977457    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:39:16.484547 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:39:16.484512    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:39:26.684252 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:39:26.684220    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:39:31.958010 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:39:31.957981    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:39:35.773121 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:39:35.773083    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:39:41.698591 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:39:41.698531    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:39:47.089736 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:39:47.089700    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:40:09.925706 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:40:09.925633    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:40:10.513079 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:40:10.513049    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:40:13.879426 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:40:13.879394    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:40:26.681522 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:40:26.681489    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:40:35.978536 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:40:35.978503    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:40:49.381097 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:40:49.381061    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:40:59.082113 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:40:59.082076    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:41:09.075254 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:41:09.075219    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:41:19.277810 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:41:19.277780    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:41:28.588822 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:41:28.588788    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:41:32.917557 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:41:32.917527    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:41:32.967890 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:41:32.967865    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:41:38.677645 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:41:38.677570    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:41:52.082850 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:41:52.082818    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:42:00.989960 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:42:00.989925    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:42:11.181597 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:42:11.181561    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:42:24.382394 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:42:24.382344    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:42:35.183877 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:42:35.183838    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:42:44.474230 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:42:44.474193    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:42:55.480745 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:42:55.480712    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:43:06.582322 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:06.582287    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:43:16.573547 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:16.573469    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:43:19.017959 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:19.017922    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:29.285108 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:29.285074    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:43:40.695924 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:40.695891    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:43.965151 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:43.965115    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:44.062925 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:44.062896    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:44.470259 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:44.470223    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:44.788258 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:44.788219    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:43:49.968031 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:49.968000    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:50.068387 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:50.068342    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:50.217893 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:50.217871    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:50.540146 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:50.540115    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:50.661471 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:50.661442    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:53.422580 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:53.422547    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:53.567065 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:53.567032    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:53.655416 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:53.655384    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:54.079352 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:54.079323    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:54.178256 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:54.178228    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:43:54.259869 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:43:54.259842    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:44:00.083309 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:00.083277    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:44:19.113699 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:19.113671    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv"]
Jul 28 04:44:19.116764 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:19.116746    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv"
Jul 28 04:44:19.118504 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:19.118486    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 04:44:19.118560 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:19.118514    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 04:44:19.123154 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:19.123128    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv"]
Jul 28 04:44:19.277008 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:19.276977    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3616ed66-17ca-462c-adf6-41afad3b3530-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv\" (UID: \"3616ed66-17ca-462c-adf6-41afad3b3530\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv"
Jul 28 04:44:19.377308 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:19.377247    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3616ed66-17ca-462c-adf6-41afad3b3530-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv\" (UID: \"3616ed66-17ca-462c-adf6-41afad3b3530\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv"
Jul 28 04:44:19.377785 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:19.377769    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3616ed66-17ca-462c-adf6-41afad3b3530-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv\" (UID: \"3616ed66-17ca-462c-adf6-41afad3b3530\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv"
Jul 28 04:44:19.429110 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:19.429087    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv"
Jul 28 04:44:19.754966 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:19.754938    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv"]
Jul 28 04:44:19.757214 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:44:19.757186    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3616ed66_17ca_462c_adf6_41afad3b3530.slice/crio-2413831038d45e205202fb8cff1da2f9b1ffaba1f2444e5a46ec7920d3f4b7ad WatchSource:0}: Error finding container 2413831038d45e205202fb8cff1da2f9b1ffaba1f2444e5a46ec7920d3f4b7ad: Status 404 returned error can't find the container with id 2413831038d45e205202fb8cff1da2f9b1ffaba1f2444e5a46ec7920d3f4b7ad
Jul 28 04:44:19.759001 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:19.758985    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 04:44:19.837439 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:19.837413    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv" event={"ID":"3616ed66-17ca-462c-adf6-41afad3b3530","Type":"ContainerStarted","Data":"2413831038d45e205202fb8cff1da2f9b1ffaba1f2444e5a46ec7920d3f4b7ad"}
Jul 28 04:44:22.855179 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:22.855094    2574 generic.go:356] "Generic (PLEG): container finished" podID="3616ed66-17ca-462c-adf6-41afad3b3530" containerID="4fa7957b182d629e7f6871b4278ac40df836b8ee718211256db96dc9ceb2bced" exitCode=0
Jul 28 04:44:22.855179 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:22.855157    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv" event={"ID":"3616ed66-17ca-462c-adf6-41afad3b3530","Type":"ContainerDied","Data":"4fa7957b182d629e7f6871b4278ac40df836b8ee718211256db96dc9ceb2bced"}
Jul 28 04:44:23.986447 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:23.986415    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv"
Jul 28 04:44:24.114047 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:24.113995    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3616ed66-17ca-462c-adf6-41afad3b3530-maas-api-service-ca\") pod \"3616ed66-17ca-462c-adf6-41afad3b3530\" (UID: \"3616ed66-17ca-462c-adf6-41afad3b3530\") "
Jul 28 04:44:24.114335 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:24.114317    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3616ed66-17ca-462c-adf6-41afad3b3530-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3616ed66-17ca-462c-adf6-41afad3b3530" (UID: "3616ed66-17ca-462c-adf6-41afad3b3530"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 04:44:24.214736 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:24.214706    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3616ed66-17ca-462c-adf6-41afad3b3530-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:44:24.865398 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:24.865352    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv" event={"ID":"3616ed66-17ca-462c-adf6-41afad3b3530","Type":"ContainerDied","Data":"2413831038d45e205202fb8cff1da2f9b1ffaba1f2444e5a46ec7920d3f4b7ad"}
Jul 28 04:44:24.865398 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:24.865392    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv"
Jul 28 04:44:24.865544 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:24.865408    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2413831038d45e205202fb8cff1da2f9b1ffaba1f2444e5a46ec7920d3f4b7ad"
Jul 28 04:44:29.666569 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:29.666541    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:44:29.730242 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:29.730216    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:44:29.783918 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:29.783893    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:44:32.575063 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:32.575030    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:44:34.201303 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:34.201264    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv"]
Jul 28 04:44:34.203662 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:34.203638    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ecc0dac1-6l5lv"]
Jul 28 04:44:34.206309 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:34.206275    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:44:35.620496 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:35.620464    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3616ed66-17ca-462c-adf6-41afad3b3530" path="/var/lib/kubelet/pods/3616ed66-17ca-462c-adf6-41afad3b3530/volumes"
Jul 28 04:44:35.640604 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:35.640580    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:44:35.848129 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:35.848107    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:44:40.448871 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:40.448800    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:44:40.571392 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:40.571346    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:44:40.804324 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:40.804295    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:44:41.164614 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:41.164544    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:44:41.263264 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:41.263234    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:44:44.082321 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:44:44.082289    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:45:00.137551 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.137520    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5"]
Jul 28 04:45:00.137926 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.137905    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3616ed66-17ca-462c-adf6-41afad3b3530" containerName="revoke-keys"
Jul 28 04:45:00.137926 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.137918    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3616ed66-17ca-462c-adf6-41afad3b3530" containerName="revoke-keys"
Jul 28 04:45:00.138031 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.137983    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3616ed66-17ca-462c-adf6-41afad3b3530" containerName="revoke-keys"
Jul 28 04:45:00.145906 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.145880    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753565-svt8q"]
Jul 28 04:45:00.146424 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.146397    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5"
Jul 28 04:45:00.150131 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.150108    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 04:45:00.151577 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.151558    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5"]
Jul 28 04:45:00.151678 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.151595    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753565-svt8q"]
Jul 28 04:45:00.151738 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.151684    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753565-svt8q"
Jul 28 04:45:00.159997 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.159975    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5"
Jul 28 04:45:00.166465 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.166452    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753565-svt8q"
Jul 28 04:45:00.296339 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.296310    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5"]
Jul 28 04:45:00.298052 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:45:00.298023    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod068223c9_2076_4e70_8e0a_7c85afb1817f.slice/crio-af934cbe835c18391fce1b3a21e43e690d4bb544f31686322703d63b15100392 WatchSource:0}: Error finding container af934cbe835c18391fce1b3a21e43e690d4bb544f31686322703d63b15100392: Status 404 returned error can't find the container with id af934cbe835c18391fce1b3a21e43e690d4bb544f31686322703d63b15100392
Jul 28 04:45:00.321188 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:00.321166    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753565-svt8q"]
Jul 28 04:45:00.323183 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:45:00.323145    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14280d8e_235e_429b_9785_ed2ff2b5fcf5.slice/crio-68fb3e63e8c423d3cc700f6aed15ca79a9a05fd6abb1ab35b3fb88db9830b933 WatchSource:0}: Error finding container 68fb3e63e8c423d3cc700f6aed15ca79a9a05fd6abb1ab35b3fb88db9830b933: Status 404 returned error can't find the container with id 68fb3e63e8c423d3cc700f6aed15ca79a9a05fd6abb1ab35b3fb88db9830b933
Jul 28 04:45:01.024518 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:01.024483    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753565-svt8q" event={"ID":"14280d8e-235e-429b-9785-ed2ff2b5fcf5","Type":"ContainerStarted","Data":"b350763c305432d2d09b499111d78facc37d451468cca24931e16b057582f670"}
Jul 28 04:45:01.024691 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:01.024540    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753565-svt8q" event={"ID":"14280d8e-235e-429b-9785-ed2ff2b5fcf5","Type":"ContainerStarted","Data":"68fb3e63e8c423d3cc700f6aed15ca79a9a05fd6abb1ab35b3fb88db9830b933"}
Jul 28 04:45:01.025988 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:01.025965    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5" event={"ID":"068223c9-2076-4e70-8e0a-7c85afb1817f","Type":"ContainerStarted","Data":"dbcdb563404dfdcec5b35360ddbce50a7901d178403b38d0295edbadd0c5b3ad"}
Jul 28 04:45:01.026063 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:01.026008    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5" event={"ID":"068223c9-2076-4e70-8e0a-7c85afb1817f","Type":"ContainerStarted","Data":"af934cbe835c18391fce1b3a21e43e690d4bb544f31686322703d63b15100392"}
Jul 28 04:45:01.038023 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:01.037987    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753565-svt8q" podStartSLOduration=1.037975005 podStartE2EDuration="1.037975005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:45:01.036923398 +0000 UTC m=+1995.978763120" watchObservedRunningTime="2026-07-28 04:45:01.037975005 +0000 UTC m=+1995.979814724"
Jul 28 04:45:01.048962 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:01.048915    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5" podStartSLOduration=1.048899849 podStartE2EDuration="1.048899849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:45:01.04850394 +0000 UTC m=+1995.990343659" watchObservedRunningTime="2026-07-28 04:45:01.048899849 +0000 UTC m=+1995.990739568"
Jul 28 04:45:02.032237 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:02.032200    2574 generic.go:356] "Generic (PLEG): container finished" podID="14280d8e-235e-429b-9785-ed2ff2b5fcf5" containerID="b350763c305432d2d09b499111d78facc37d451468cca24931e16b057582f670" exitCode=0
Jul 28 04:45:02.032623 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:02.032285    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753565-svt8q" event={"ID":"14280d8e-235e-429b-9785-ed2ff2b5fcf5","Type":"ContainerDied","Data":"b350763c305432d2d09b499111d78facc37d451468cca24931e16b057582f670"}
Jul 28 04:45:02.033738 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:02.033716    2574 generic.go:356] "Generic (PLEG): container finished" podID="068223c9-2076-4e70-8e0a-7c85afb1817f" containerID="dbcdb563404dfdcec5b35360ddbce50a7901d178403b38d0295edbadd0c5b3ad" exitCode=0
Jul 28 04:45:02.033838 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:02.033779    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5" event={"ID":"068223c9-2076-4e70-8e0a-7c85afb1817f","Type":"ContainerDied","Data":"dbcdb563404dfdcec5b35360ddbce50a7901d178403b38d0295edbadd0c5b3ad"}
Jul 28 04:45:03.178379 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:03.178341    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5"
Jul 28 04:45:03.181162 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:03.181143    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753565-svt8q"
Jul 28 04:45:04.044355 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:04.044335    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5"
Jul 28 04:45:04.044500 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:04.044399    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5" event={"ID":"068223c9-2076-4e70-8e0a-7c85afb1817f","Type":"ContainerDied","Data":"af934cbe835c18391fce1b3a21e43e690d4bb544f31686322703d63b15100392"}
Jul 28 04:45:04.044500 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:04.044454    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af934cbe835c18391fce1b3a21e43e690d4bb544f31686322703d63b15100392"
Jul 28 04:45:04.045617 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:04.045593    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753565-svt8q" event={"ID":"14280d8e-235e-429b-9785-ed2ff2b5fcf5","Type":"ContainerDied","Data":"68fb3e63e8c423d3cc700f6aed15ca79a9a05fd6abb1ab35b3fb88db9830b933"}
Jul 28 04:45:04.045710 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:04.045624    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68fb3e63e8c423d3cc700f6aed15ca79a9a05fd6abb1ab35b3fb88db9830b933"
Jul 28 04:45:04.045710 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:04.045597    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753565-svt8q"
Jul 28 04:45:05.637695 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.637659    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd"]
Jul 28 04:45:05.638181 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.638161    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14280d8e-235e-429b-9785-ed2ff2b5fcf5" containerName="cleanup"
Jul 28 04:45:05.638273 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.638183    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="14280d8e-235e-429b-9785-ed2ff2b5fcf5" containerName="cleanup"
Jul 28 04:45:05.638273 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.638219    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="068223c9-2076-4e70-8e0a-7c85afb1817f" containerName="cleanup"
Jul 28 04:45:05.638273 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.638227    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="068223c9-2076-4e70-8e0a-7c85afb1817f" containerName="cleanup"
Jul 28 04:45:05.638438 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.638322    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="14280d8e-235e-429b-9785-ed2ff2b5fcf5" containerName="cleanup"
Jul 28 04:45:05.638438 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.638335    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="068223c9-2076-4e70-8e0a-7c85afb1817f" containerName="cleanup"
Jul 28 04:45:05.641381 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.641338    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd"
Jul 28 04:45:05.643432 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.643411    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 04:45:05.643532 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.643471    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 04:45:05.647304 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.647270    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd"]
Jul 28 04:45:05.721229 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.721205    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04504ed5-6c86-4192-87e5-003990ca886f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd\" (UID: \"04504ed5-6c86-4192-87e5-003990ca886f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd"
Jul 28 04:45:05.821749 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.821725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04504ed5-6c86-4192-87e5-003990ca886f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd\" (UID: \"04504ed5-6c86-4192-87e5-003990ca886f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd"
Jul 28 04:45:05.822321 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.822305    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04504ed5-6c86-4192-87e5-003990ca886f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd\" (UID: \"04504ed5-6c86-4192-87e5-003990ca886f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd"
Jul 28 04:45:05.952639 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:05.952594    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd"
Jul 28 04:45:06.281246 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:06.281220    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd"]
Jul 28 04:45:06.283051 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:45:06.283020    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04504ed5_6c86_4192_87e5_003990ca886f.slice/crio-c19c349a0e0aa8bd8e92f029f0bb6036cc14350b544c8c933a0d60ce369867fd WatchSource:0}: Error finding container c19c349a0e0aa8bd8e92f029f0bb6036cc14350b544c8c933a0d60ce369867fd: Status 404 returned error can't find the container with id c19c349a0e0aa8bd8e92f029f0bb6036cc14350b544c8c933a0d60ce369867fd
Jul 28 04:45:07.060467 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:07.060443    2574 generic.go:356] "Generic (PLEG): container finished" podID="04504ed5-6c86-4192-87e5-003990ca886f" containerID="fa65563aaee06952ce1b9219df5c4f158cc9351599a12e612a16b061c79fab25" exitCode=0
Jul 28 04:45:07.060737 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:07.060501    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd" event={"ID":"04504ed5-6c86-4192-87e5-003990ca886f","Type":"ContainerDied","Data":"fa65563aaee06952ce1b9219df5c4f158cc9351599a12e612a16b061c79fab25"}
Jul 28 04:45:07.060737 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:07.060536    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd" event={"ID":"04504ed5-6c86-4192-87e5-003990ca886f","Type":"ContainerStarted","Data":"c19c349a0e0aa8bd8e92f029f0bb6036cc14350b544c8c933a0d60ce369867fd"}
Jul 28 04:45:08.190141 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:08.190118    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd"
Jul 28 04:45:08.239527 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:08.239505    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04504ed5-6c86-4192-87e5-003990ca886f-maas-api-service-ca\") pod \"04504ed5-6c86-4192-87e5-003990ca886f\" (UID: \"04504ed5-6c86-4192-87e5-003990ca886f\") "
Jul 28 04:45:08.239804 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:08.239785    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04504ed5-6c86-4192-87e5-003990ca886f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "04504ed5-6c86-4192-87e5-003990ca886f" (UID: "04504ed5-6c86-4192-87e5-003990ca886f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 04:45:08.340563 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:08.340513    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04504ed5-6c86-4192-87e5-003990ca886f-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:45:09.071611 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:09.071569    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd" event={"ID":"04504ed5-6c86-4192-87e5-003990ca886f","Type":"ContainerDied","Data":"c19c349a0e0aa8bd8e92f029f0bb6036cc14350b544c8c933a0d60ce369867fd"}
Jul 28 04:45:09.071611 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:09.071596    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd"
Jul 28 04:45:09.071795 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:09.071625    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c19c349a0e0aa8bd8e92f029f0bb6036cc14350b544c8c933a0d60ce369867fd"
Jul 28 04:45:16.095948 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:16.095893    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5"]
Jul 28 04:45:16.096514 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:16.096042    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:16.098807 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:16.098784    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5f6a9856-29753565-wq9j5"]
Jul 28 04:45:16.162816 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:16.162782    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:16.220882 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:16.220854    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:17.620077 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:17.620042    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="068223c9-2076-4e70-8e0a-7c85afb1817f" path="/var/lib/kubelet/pods/068223c9-2076-4e70-8e0a-7c85afb1817f/volumes"
Jul 28 04:45:19.384669 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:19.384636    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:45:20.714646 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:20.714614    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:20.719040 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:20.719012    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd"]
Jul 28 04:45:20.721060 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:20.721037    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5f6a9856-s22zd"]
Jul 28 04:45:20.911848 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:20.911818    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:21.177455 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:21.177430    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:21.428981 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:21.428918    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:21.619964 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:21.619942    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04504ed5-6c86-4192-87e5-003990ca886f" path="/var/lib/kubelet/pods/04504ed5-6c86-4192-87e5-003990ca886f/volumes"
Jul 28 04:45:21.753831 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:21.753762    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:22.823709 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:22.823682    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:24.560666 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:24.560635    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:24.655546 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:24.655519    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:24.736669 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:24.736637    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:25.249466 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:25.249440    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:25.361025 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:25.360997    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:25.469506 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:25.469477    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:45:30.176221 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:30.176185    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:45:50.549608 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:50.549575    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6"]
Jul 28 04:45:50.550047 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:50.549903    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04504ed5-6c86-4192-87e5-003990ca886f" containerName="revoke-keys"
Jul 28 04:45:50.550047 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:50.549914    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="04504ed5-6c86-4192-87e5-003990ca886f" containerName="revoke-keys"
Jul 28 04:45:50.550047 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:50.549989    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="04504ed5-6c86-4192-87e5-003990ca886f" containerName="revoke-keys"
Jul 28 04:45:50.552844 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:50.552826    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6"
Jul 28 04:45:50.554998 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:50.554978    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 04:45:50.555122 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:50.555013    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 04:45:50.558616 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:50.558574    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6"]
Jul 28 04:45:50.656754 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:50.656727    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/241e2abe-a655-4571-9070-b09963ed03e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6\" (UID: \"241e2abe-a655-4571-9070-b09963ed03e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6"
Jul 28 04:45:50.758052 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:50.758026    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/241e2abe-a655-4571-9070-b09963ed03e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6\" (UID: \"241e2abe-a655-4571-9070-b09963ed03e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6"
Jul 28 04:45:50.758654 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:50.758635    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/241e2abe-a655-4571-9070-b09963ed03e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6\" (UID: \"241e2abe-a655-4571-9070-b09963ed03e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6"
Jul 28 04:45:50.863971 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:50.863924    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6"
Jul 28 04:45:50.985725 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:50.985699    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6"]
Jul 28 04:45:50.988321 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:45:50.988297    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241e2abe_a655_4571_9070_b09963ed03e5.slice/crio-7aadbb81cbbad4b2afc31fc9a9bbc30b1898fadb74d9324db0c05525076d8bba WatchSource:0}: Error finding container 7aadbb81cbbad4b2afc31fc9a9bbc30b1898fadb74d9324db0c05525076d8bba: Status 404 returned error can't find the container with id 7aadbb81cbbad4b2afc31fc9a9bbc30b1898fadb74d9324db0c05525076d8bba
Jul 28 04:45:51.274413 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:51.274375    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6" event={"ID":"241e2abe-a655-4571-9070-b09963ed03e5","Type":"ContainerStarted","Data":"20b5f1712089b1f14f0967b9bebbbec8d8a73d53ed34d09492753a252f0ea09d"}
Jul 28 04:45:51.274516 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:51.274473    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6" event={"ID":"241e2abe-a655-4571-9070-b09963ed03e5","Type":"ContainerStarted","Data":"7aadbb81cbbad4b2afc31fc9a9bbc30b1898fadb74d9324db0c05525076d8bba"}
Jul 28 04:45:51.289201 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:51.289163    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6" podStartSLOduration=1.289149194 podStartE2EDuration="1.289149194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:45:51.288127477 +0000 UTC m=+2046.229967230" watchObservedRunningTime="2026-07-28 04:45:51.289149194 +0000 UTC m=+2046.230988951"
Jul 28 04:45:52.280662 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:52.280632    2574 generic.go:356] "Generic (PLEG): container finished" podID="241e2abe-a655-4571-9070-b09963ed03e5" containerID="20b5f1712089b1f14f0967b9bebbbec8d8a73d53ed34d09492753a252f0ea09d" exitCode=0
Jul 28 04:45:52.281024 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:52.280717    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6" event={"ID":"241e2abe-a655-4571-9070-b09963ed03e5","Type":"ContainerDied","Data":"20b5f1712089b1f14f0967b9bebbbec8d8a73d53ed34d09492753a252f0ea09d"}
Jul 28 04:45:53.414379 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:53.414338    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6"
Jul 28 04:45:53.578270 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:53.578206    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/241e2abe-a655-4571-9070-b09963ed03e5-maas-api-service-ca\") pod \"241e2abe-a655-4571-9070-b09963ed03e5\" (UID: \"241e2abe-a655-4571-9070-b09963ed03e5\") "
Jul 28 04:45:53.578594 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:53.578573    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241e2abe-a655-4571-9070-b09963ed03e5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "241e2abe-a655-4571-9070-b09963ed03e5" (UID: "241e2abe-a655-4571-9070-b09963ed03e5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 04:45:53.679143 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:53.679120    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/241e2abe-a655-4571-9070-b09963ed03e5-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:45:54.291409 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:54.291384    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6"
Jul 28 04:45:54.291409 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:54.291393    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6" event={"ID":"241e2abe-a655-4571-9070-b09963ed03e5","Type":"ContainerDied","Data":"7aadbb81cbbad4b2afc31fc9a9bbc30b1898fadb74d9324db0c05525076d8bba"}
Jul 28 04:45:54.291610 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:45:54.291434    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7aadbb81cbbad4b2afc31fc9a9bbc30b1898fadb74d9324db0c05525076d8bba"
Jul 28 04:46:00.993677 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:00.993638    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:01.094376 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:01.094334    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:01.194248 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:01.194219    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:04.078613 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:04.078579    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:46:05.622745 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:05.622713    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6"]
Jul 28 04:46:05.623229 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:05.623206    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:05.626246 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:05.626225    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b42b78c0-ch7x6"]
Jul 28 04:46:06.923248 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:06.923221    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:07.174128 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:07.174047    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:07.429530 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:07.429440    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:07.620324 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:07.620285    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="241e2abe-a655-4571-9070-b09963ed03e5" path="/var/lib/kubelet/pods/241e2abe-a655-4571-9070-b09963ed03e5/volumes"
Jul 28 04:46:07.760001 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:07.759929    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:08.439324 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:08.439289    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:09.852429 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:09.852399    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:09.950473 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:09.950444    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:10.264742 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:10.264671    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:10.453614 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:10.453584    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:10.734069 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:10.734041    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:10.833945 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:10.833915    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:15.473706 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:15.473670    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:46:19.461323 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:19.461291    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz"]
Jul 28 04:46:19.461694 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:19.461639    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241e2abe-a655-4571-9070-b09963ed03e5" containerName="revoke-keys"
Jul 28 04:46:19.461694 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:19.461649    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="241e2abe-a655-4571-9070-b09963ed03e5" containerName="revoke-keys"
Jul 28 04:46:19.461777 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:19.461712    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="241e2abe-a655-4571-9070-b09963ed03e5" containerName="revoke-keys"
Jul 28 04:46:19.464587 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:19.464570    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz"
Jul 28 04:46:19.466607 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:19.466589    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 04:46:19.466665 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:19.466632    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 04:46:19.470674 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:19.470649    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz"]
Jul 28 04:46:19.568860 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:19.568838    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccc2be45-e80c-4078-a720-650db4f2ec48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz\" (UID: \"ccc2be45-e80c-4078-a720-650db4f2ec48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz"
Jul 28 04:46:19.669859 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:19.669831    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccc2be45-e80c-4078-a720-650db4f2ec48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz\" (UID: \"ccc2be45-e80c-4078-a720-650db4f2ec48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz"
Jul 28 04:46:19.670379 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:19.670343    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccc2be45-e80c-4078-a720-650db4f2ec48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz\" (UID: \"ccc2be45-e80c-4078-a720-650db4f2ec48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz"
Jul 28 04:46:19.775774 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:19.775758    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz"
Jul 28 04:46:19.894486 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:19.894463    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz"]
Jul 28 04:46:19.896223 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:46:19.896197    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccc2be45_e80c_4078_a720_650db4f2ec48.slice/crio-5f82cfe706eaef780c07a6fb86fa5e4249818359612baaa3fd288f2b6fc5f487 WatchSource:0}: Error finding container 5f82cfe706eaef780c07a6fb86fa5e4249818359612baaa3fd288f2b6fc5f487: Status 404 returned error can't find the container with id 5f82cfe706eaef780c07a6fb86fa5e4249818359612baaa3fd288f2b6fc5f487
Jul 28 04:46:20.415728 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:20.415675    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz" event={"ID":"ccc2be45-e80c-4078-a720-650db4f2ec48","Type":"ContainerStarted","Data":"8d004006f06e251cb2a1daf821fc3a1f7e33ec4d3166aa25afa7cdf1799db92a"}
Jul 28 04:46:20.415825 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:20.415733    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz" event={"ID":"ccc2be45-e80c-4078-a720-650db4f2ec48","Type":"ContainerStarted","Data":"5f82cfe706eaef780c07a6fb86fa5e4249818359612baaa3fd288f2b6fc5f487"}
Jul 28 04:46:20.430432 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:20.430390    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz" podStartSLOduration=1.430376795 podStartE2EDuration="1.430376795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:46:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:46:20.429131206 +0000 UTC m=+2075.370970926" watchObservedRunningTime="2026-07-28 04:46:20.430376795 +0000 UTC m=+2075.372216504"
Jul 28 04:46:21.422516 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:21.422483    2574 generic.go:356] "Generic (PLEG): container finished" podID="ccc2be45-e80c-4078-a720-650db4f2ec48" containerID="8d004006f06e251cb2a1daf821fc3a1f7e33ec4d3166aa25afa7cdf1799db92a" exitCode=0
Jul 28 04:46:21.422876 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:21.422568    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz" event={"ID":"ccc2be45-e80c-4078-a720-650db4f2ec48","Type":"ContainerDied","Data":"8d004006f06e251cb2a1daf821fc3a1f7e33ec4d3166aa25afa7cdf1799db92a"}
Jul 28 04:46:22.553450 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:22.553427    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz"
Jul 28 04:46:22.590902 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:22.590879    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccc2be45-e80c-4078-a720-650db4f2ec48-maas-api-service-ca\") pod \"ccc2be45-e80c-4078-a720-650db4f2ec48\" (UID: \"ccc2be45-e80c-4078-a720-650db4f2ec48\") "
Jul 28 04:46:22.591174 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:22.591156    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccc2be45-e80c-4078-a720-650db4f2ec48-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ccc2be45-e80c-4078-a720-650db4f2ec48" (UID: "ccc2be45-e80c-4078-a720-650db4f2ec48"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 04:46:22.692352 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:22.692299    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccc2be45-e80c-4078-a720-650db4f2ec48-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:46:23.433559 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:23.433527    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz"
Jul 28 04:46:23.433708 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:23.433558    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz" event={"ID":"ccc2be45-e80c-4078-a720-650db4f2ec48","Type":"ContainerDied","Data":"5f82cfe706eaef780c07a6fb86fa5e4249818359612baaa3fd288f2b6fc5f487"}
Jul 28 04:46:23.433708 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:23.433595    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f82cfe706eaef780c07a6fb86fa5e4249818359612baaa3fd288f2b6fc5f487"
Jul 28 04:46:29.775371 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:29.775318    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:29.876012 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:29.875979    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:29.975506 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:29.975479    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:32.680227 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:32.680196    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:46:34.531539 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:34.531510    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:34.537248 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:34.537217    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz"]
Jul 28 04:46:34.538759 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:34.538743    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ac8148f2-ctchz"]
Jul 28 04:46:34.797602 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:34.797533    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:34.980142 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:34.980100    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:35.486760 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:35.486728    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:35.587008 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:35.586982    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:35.620450 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:35.620426    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccc2be45-e80c-4078-a720-650db4f2ec48" path="/var/lib/kubelet/pods/ccc2be45-e80c-4078-a720-650db4f2ec48/volumes"
Jul 28 04:46:35.691299 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:35.691271    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:37.201515 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:37.201484    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:42.868470 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:42.868442    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:42.920211 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:42.920184    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:43.012004 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:43.011982    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:43.113990 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:43.113958    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:43.285541 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:43.285505    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:43.684659 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:43.684592    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:46:55.144638 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:46:55.144605    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:09.742292 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:09.742257    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:16.023926 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:16.023891    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:16.127714 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:16.127689    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:16.223824 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:16.223795    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:32.206732 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:32.206698    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:37.937147 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:37.937058    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:37.993702 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:37.993672    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:38.100616 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:38.100555    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:38.198561 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:38.198496    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:48.695577 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:48.695542    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:49.549204 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:49.549170    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:56.139407 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:56.139379    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:56.242486 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:56.242460    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:56.338500 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:56.338471    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:56.438417 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:56.438342    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:47:59.974888 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:47:59.974858    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:48:10.190226 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:10.190188    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:10.289638 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:10.289605    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:10.389578 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:10.389553    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:12.309703 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:12.309659    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:15.068868 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:15.068825    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:15.485672 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:15.485600    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:48:18.115733 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:18.115699    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:18.215553 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:18.215525    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:18.858993 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:18.858962    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:24.520931 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:24.520904    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:24.585387 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:24.585339    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:24.787152 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:24.787131    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:48:24.789694 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:24.789673    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:25.149433 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:25.149336    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:25.354989 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:25.354957    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:28.140693 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:28.140661    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:28.237218 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:28.237191    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:28.416814 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:28.416752    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:28.564794 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:28.564766    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:28.896492 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:28.896459    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:28.996155 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:28.996128    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:32.636570 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:32.636536    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:33.095447 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:33.095416    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:48:33.324297 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:33.324250    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:33.420084 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:33.420020    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:33.771554 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:33.771482    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:39.364692 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:39.364657    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:39.464440 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:39.464407    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:39.736927 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:39.736865    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw"]
Jul 28 04:48:39.737208 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:39.737196    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccc2be45-e80c-4078-a720-650db4f2ec48" containerName="revoke-keys"
Jul 28 04:48:39.737249 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:39.737210    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccc2be45-e80c-4078-a720-650db4f2ec48" containerName="revoke-keys"
Jul 28 04:48:39.737280 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:39.737268    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccc2be45-e80c-4078-a720-650db4f2ec48" containerName="revoke-keys"
Jul 28 04:48:39.740207 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:39.740189    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw"
Jul 28 04:48:39.742800 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:39.742770    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 04:48:39.742916 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:39.742901    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 04:48:39.749006 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:39.748982    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw"]
Jul 28 04:48:39.862563 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:39.862535    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/978b11ce-560c-4966-8db3-eb7ff04e3cfd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw\" (UID: \"978b11ce-560c-4966-8db3-eb7ff04e3cfd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw"
Jul 28 04:48:39.963575 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:39.963551    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/978b11ce-560c-4966-8db3-eb7ff04e3cfd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw\" (UID: \"978b11ce-560c-4966-8db3-eb7ff04e3cfd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw"
Jul 28 04:48:39.964092 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:39.964076    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/978b11ce-560c-4966-8db3-eb7ff04e3cfd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw\" (UID: \"978b11ce-560c-4966-8db3-eb7ff04e3cfd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw"
Jul 28 04:48:40.050152 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:40.050124    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw"
Jul 28 04:48:40.172616 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:40.172587    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw"]
Jul 28 04:48:40.174652 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:48:40.174630    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod978b11ce_560c_4966_8db3_eb7ff04e3cfd.slice/crio-a0f4636ec2672d622369ed14e22c0b700171e25f543cd4eeb8d1cfa13886c100 WatchSource:0}: Error finding container a0f4636ec2672d622369ed14e22c0b700171e25f543cd4eeb8d1cfa13886c100: Status 404 returned error can't find the container with id a0f4636ec2672d622369ed14e22c0b700171e25f543cd4eeb8d1cfa13886c100
Jul 28 04:48:41.074703 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:41.074677    2574 generic.go:356] "Generic (PLEG): container finished" podID="978b11ce-560c-4966-8db3-eb7ff04e3cfd" containerID="8e3e9efdee8edda7ca739b1b026309b2ed7acd971c4d667694a74d23a8313537" exitCode=0
Jul 28 04:48:41.075019 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:41.074741    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw" event={"ID":"978b11ce-560c-4966-8db3-eb7ff04e3cfd","Type":"ContainerDied","Data":"8e3e9efdee8edda7ca739b1b026309b2ed7acd971c4d667694a74d23a8313537"}
Jul 28 04:48:41.075019 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:41.074775    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw" event={"ID":"978b11ce-560c-4966-8db3-eb7ff04e3cfd","Type":"ContainerStarted","Data":"a0f4636ec2672d622369ed14e22c0b700171e25f543cd4eeb8d1cfa13886c100"}
Jul 28 04:48:42.204660 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:42.204637    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw"
Jul 28 04:48:42.282370 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:42.282343    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/978b11ce-560c-4966-8db3-eb7ff04e3cfd-maas-api-service-ca\") pod \"978b11ce-560c-4966-8db3-eb7ff04e3cfd\" (UID: \"978b11ce-560c-4966-8db3-eb7ff04e3cfd\") "
Jul 28 04:48:42.282690 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:42.282672    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/978b11ce-560c-4966-8db3-eb7ff04e3cfd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "978b11ce-560c-4966-8db3-eb7ff04e3cfd" (UID: "978b11ce-560c-4966-8db3-eb7ff04e3cfd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 04:48:42.383111 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:42.383059    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/978b11ce-560c-4966-8db3-eb7ff04e3cfd-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:48:43.085325 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:43.085299    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw"
Jul 28 04:48:43.085494 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:43.085299    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw" event={"ID":"978b11ce-560c-4966-8db3-eb7ff04e3cfd","Type":"ContainerDied","Data":"a0f4636ec2672d622369ed14e22c0b700171e25f543cd4eeb8d1cfa13886c100"}
Jul 28 04:48:43.085494 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:43.085418    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0f4636ec2672d622369ed14e22c0b700171e25f543cd4eeb8d1cfa13886c100"
Jul 28 04:48:50.189904 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:50.189868    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:50.262321 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:50.262288    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:50.319725 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:50.319696    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:53.388482 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:53.388451    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:48:54.817262 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:54.817229    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw"]
Jul 28 04:48:54.817751 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:54.817679    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:54.819374 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:54.819337    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ae9f1f22-h5dxw"]
Jul 28 04:48:55.211714 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:55.211628    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:48:55.619996 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:48:55.619953    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="978b11ce-560c-4966-8db3-eb7ff04e3cfd" path="/var/lib/kubelet/pods/978b11ce-560c-4966-8db3-eb7ff04e3cfd/volumes"
Jul 28 04:49:00.793522 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:00.793486    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:00.870037 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:00.870012    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:01.071576 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:01.071503    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:01.321114 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:01.321078    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:01.460738 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:01.460611    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:01.557515 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:01.557471    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:01.682438 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:01.682392    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:07.312848 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:07.312819    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:07.374268 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:07.374185    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:07.457643 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:07.457612    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:07.559803 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:07.559774    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:07.697426 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:07.697348    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:13.328571 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:13.328544    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:13.396823 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:13.396792    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:20.473335 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:20.473297    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:20.791439 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:20.791409    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:21.023935 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:21.023908    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:23.458090 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:23.458059    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:23.557205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:23.557173    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:23.710065 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:23.709987    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:23.816991 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:23.816958    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:24.248650 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:24.248614    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:24.347293 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:24.347261    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:24.420966 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:24.420932    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:27.986453 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:27.986419    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:30.074067 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:30.074030    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:49:38.983830 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:38.983799    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"]
Jul 28 04:49:38.984284 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:38.984158    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="978b11ce-560c-4966-8db3-eb7ff04e3cfd" containerName="revoke-keys"
Jul 28 04:49:38.984284 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:38.984168    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="978b11ce-560c-4966-8db3-eb7ff04e3cfd" containerName="revoke-keys"
Jul 28 04:49:38.984284 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:38.984235    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="978b11ce-560c-4966-8db3-eb7ff04e3cfd" containerName="revoke-keys"
Jul 28 04:49:38.987413 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:38.987391    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:38.990102 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:38.990078    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-890677f3\"/\"kube-root-ca.crt\""
Jul 28 04:49:38.990218 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:38.990085    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-890677f3\"/\"openshift-service-ca.crt\""
Jul 28 04:49:38.996078 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:38.990639    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-890677f3\"/\"default-dockercfg-zbxwp\""
Jul 28 04:49:38.996078 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:38.990656    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-890677f3\"/\"e2e-lifecycle-model-890677f3-kserve-self-signed-certs\""
Jul 28 04:49:39.000612 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.000012    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"]
Jul 28 04:49:39.008719 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.008691    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-tmp-dir\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.008840 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.008780    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-model-cache\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.008840 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.008811    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-home\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.008955 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.008900    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-dshm\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.009009 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.008951    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjd8l\" (UniqueName: \"kubernetes.io/projected/d0df09e2-f325-4947-b093-8788c49c01a6-kube-api-access-kjd8l\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.009009 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.008979    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0df09e2-f325-4947-b093-8788c49c01a6-tls-certs\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.014971 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.014951    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:49:39.109985 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.109964    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-model-cache\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.110099 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.109995    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-home\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.110099 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.110018    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-dshm\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.110099 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.110045    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjd8l\" (UniqueName: \"kubernetes.io/projected/d0df09e2-f325-4947-b093-8788c49c01a6-kube-api-access-kjd8l\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.110099 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.110069    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0df09e2-f325-4947-b093-8788c49c01a6-tls-certs\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.110303 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.110108    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-tmp-dir\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.110393 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.110348    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-home\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.110511 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.110492    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-model-cache\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.110572 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.110498    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-tmp-dir\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.112239 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.112220    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-dshm\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.112430 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.112411    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0df09e2-f325-4947-b093-8788c49c01a6-tls-certs\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.117029 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.117009    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjd8l\" (UniqueName: \"kubernetes.io/projected/d0df09e2-f325-4947-b093-8788c49c01a6-kube-api-access-kjd8l\") pod \"e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") " pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.303289 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.303268    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:39.428048 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:39.428024    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"]
Jul 28 04:49:39.430449 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:49:39.430421    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0df09e2_f325_4947_b093_8788c49c01a6.slice/crio-9afcc69473e8a92bd4491296d196ad67c3eff4c804778f31039577f1363b9c24 WatchSource:0}: Error finding container 9afcc69473e8a92bd4491296d196ad67c3eff4c804778f31039577f1363b9c24: Status 404 returned error can't find the container with id 9afcc69473e8a92bd4491296d196ad67c3eff4c804778f31039577f1363b9c24
Jul 28 04:49:40.360730 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:40.360697    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp" event={"ID":"d0df09e2-f325-4947-b093-8788c49c01a6","Type":"ContainerStarted","Data":"5a1a9eaceb42c1b9fd555affdf4c2112a4551a6173a74057d4c316d8c614e382"}
Jul 28 04:49:40.361065 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:40.360739    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp" event={"ID":"d0df09e2-f325-4947-b093-8788c49c01a6","Type":"ContainerStarted","Data":"9afcc69473e8a92bd4491296d196ad67c3eff4c804778f31039577f1363b9c24"}
Jul 28 04:49:40.375660 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:40.375617    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp" podStartSLOduration=2.375601961 podStartE2EDuration="2.375601961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:49:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:49:40.375255274 +0000 UTC m=+2275.317095005" watchObservedRunningTime="2026-07-28 04:49:40.375601961 +0000 UTC m=+2275.317441679"
Jul 28 04:49:42.376819 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:42.376786    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:49:49.303727 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:49.303688    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:49.304186 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:49.303938    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:49.315938 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:49.315911    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:49:49.416241 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:49:49.416218    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:50:03.279832 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:03.279803    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:50:07.643291 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:07.643259    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4"]
Jul 28 04:50:07.646596 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:07.646578    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4"
Jul 28 04:50:07.648479 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:07.648458    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 04:50:07.648479 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:07.648473    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 04:50:07.652637 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:07.652615    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4"]
Jul 28 04:50:07.721670 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:07.721637    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ebfe7d1-c1ef-4934-ac25-619d9673e000-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-890677f3-tncs4\" (UID: \"4ebfe7d1-c1ef-4934-ac25-619d9673e000\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4"
Jul 28 04:50:07.822713 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:07.822685    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ebfe7d1-c1ef-4934-ac25-619d9673e000-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-890677f3-tncs4\" (UID: \"4ebfe7d1-c1ef-4934-ac25-619d9673e000\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4"
Jul 28 04:50:07.823189 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:07.823171    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ebfe7d1-c1ef-4934-ac25-619d9673e000-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-890677f3-tncs4\" (UID: \"4ebfe7d1-c1ef-4934-ac25-619d9673e000\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4"
Jul 28 04:50:07.958550 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:07.958500    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4"
Jul 28 04:50:08.078668 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:08.078584    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4"]
Jul 28 04:50:08.081004 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:50:08.080978    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ebfe7d1_c1ef_4934_ac25_619d9673e000.slice/crio-c3a23d34667258da28fa4aea29ed1d90f9da9c323262e00bc80b718ee03f33b1 WatchSource:0}: Error finding container c3a23d34667258da28fa4aea29ed1d90f9da9c323262e00bc80b718ee03f33b1: Status 404 returned error can't find the container with id c3a23d34667258da28fa4aea29ed1d90f9da9c323262e00bc80b718ee03f33b1
Jul 28 04:50:08.499333 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:08.499266    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4" event={"ID":"4ebfe7d1-c1ef-4934-ac25-619d9673e000","Type":"ContainerStarted","Data":"f461556ba1223fd5f282dfefc5a2728ca251b5a7dfab20817c84c45f11df56d3"}
Jul 28 04:50:08.499469 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:08.499339    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4" event={"ID":"4ebfe7d1-c1ef-4934-ac25-619d9673e000","Type":"ContainerStarted","Data":"c3a23d34667258da28fa4aea29ed1d90f9da9c323262e00bc80b718ee03f33b1"}
Jul 28 04:50:08.514716 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:08.514673    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4" podStartSLOduration=1.514657081 podStartE2EDuration="1.514657081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:50:08.512699089 +0000 UTC m=+2303.454538810" watchObservedRunningTime="2026-07-28 04:50:08.514657081 +0000 UTC m=+2303.456496802"
Jul 28 04:50:09.506287 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:09.506255    2574 generic.go:356] "Generic (PLEG): container finished" podID="4ebfe7d1-c1ef-4934-ac25-619d9673e000" containerID="f461556ba1223fd5f282dfefc5a2728ca251b5a7dfab20817c84c45f11df56d3" exitCode=0
Jul 28 04:50:09.506671 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:09.506328    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4" event={"ID":"4ebfe7d1-c1ef-4934-ac25-619d9673e000","Type":"ContainerDied","Data":"f461556ba1223fd5f282dfefc5a2728ca251b5a7dfab20817c84c45f11df56d3"}
Jul 28 04:50:10.644521 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:10.644492    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4"
Jul 28 04:50:10.741968 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:10.741946    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ebfe7d1-c1ef-4934-ac25-619d9673e000-maas-api-service-ca\") pod \"4ebfe7d1-c1ef-4934-ac25-619d9673e000\" (UID: \"4ebfe7d1-c1ef-4934-ac25-619d9673e000\") "
Jul 28 04:50:10.742288 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:10.742267    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ebfe7d1-c1ef-4934-ac25-619d9673e000-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ebfe7d1-c1ef-4934-ac25-619d9673e000" (UID: "4ebfe7d1-c1ef-4934-ac25-619d9673e000"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 04:50:10.842986 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:10.842967    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ebfe7d1-c1ef-4934-ac25-619d9673e000-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:50:11.519067 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:11.519043    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4"
Jul 28 04:50:11.519225 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:11.519088    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4" event={"ID":"4ebfe7d1-c1ef-4934-ac25-619d9673e000","Type":"ContainerDied","Data":"c3a23d34667258da28fa4aea29ed1d90f9da9c323262e00bc80b718ee03f33b1"}
Jul 28 04:50:11.519225 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:11.519131    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3a23d34667258da28fa4aea29ed1d90f9da9c323262e00bc80b718ee03f33b1"
Jul 28 04:50:13.825772 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:13.825719    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:50:15.431564 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:15.431512    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:50:21.783475 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:21.783440    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:50:23.371610 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:23.371577    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:50:23.438390 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:23.438320    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:50:23.496930 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:23.496907    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:50:27.730862 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:27.730831    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4"]
Jul 28 04:50:27.731385 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:27.731304    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:50:27.733059 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:27.733038    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890677f3-tncs4"]
Jul 28 04:50:29.620552 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:29.620522    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ebfe7d1-c1ef-4934-ac25-619d9673e000" path="/var/lib/kubelet/pods/4ebfe7d1-c1ef-4934-ac25-619d9673e000/volumes"
Jul 28 04:50:31.113242 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:31.113205    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:50:32.485595 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:32.485426    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:50:36.095556 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:36.095528    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"]
Jul 28 04:50:36.137924 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:36.137891    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:50:45.665736 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:45.665678    2574 scope.go:117] "RemoveContainer" containerID="4fa7957b182d629e7f6871b4278ac40df836b8ee718211256db96dc9ceb2bced"
Jul 28 04:50:51.114238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.114186    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp" podUID="d0df09e2-f325-4947-b093-8788c49c01a6" containerName="main" containerID="cri-o://5a1a9eaceb42c1b9fd555affdf4c2112a4551a6173a74057d4c316d8c614e382" gracePeriod=45
Jul 28 04:50:51.363160 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.363139    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:50:51.539092 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.539066    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-tmp-dir\") pod \"d0df09e2-f325-4947-b093-8788c49c01a6\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") "
Jul 28 04:50:51.539226 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.539104    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-home\") pod \"d0df09e2-f325-4947-b093-8788c49c01a6\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") "
Jul 28 04:50:51.539226 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.539130    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0df09e2-f325-4947-b093-8788c49c01a6-tls-certs\") pod \"d0df09e2-f325-4947-b093-8788c49c01a6\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") "
Jul 28 04:50:51.539226 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.539163    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-dshm\") pod \"d0df09e2-f325-4947-b093-8788c49c01a6\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") "
Jul 28 04:50:51.539226 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.539180    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-model-cache\") pod \"d0df09e2-f325-4947-b093-8788c49c01a6\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") "
Jul 28 04:50:51.539226 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.539211    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjd8l\" (UniqueName: \"kubernetes.io/projected/d0df09e2-f325-4947-b093-8788c49c01a6-kube-api-access-kjd8l\") pod \"d0df09e2-f325-4947-b093-8788c49c01a6\" (UID: \"d0df09e2-f325-4947-b093-8788c49c01a6\") "
Jul 28 04:50:51.539515 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.539346    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-home" (OuterVolumeSpecName: "home") pod "d0df09e2-f325-4947-b093-8788c49c01a6" (UID: "d0df09e2-f325-4947-b093-8788c49c01a6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 04:50:51.539515 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.539375    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0df09e2-f325-4947-b093-8788c49c01a6" (UID: "d0df09e2-f325-4947-b093-8788c49c01a6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 04:50:51.539515 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.539459    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0df09e2-f325-4947-b093-8788c49c01a6" (UID: "d0df09e2-f325-4947-b093-8788c49c01a6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 04:50:51.539515 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.539504    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-home\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:50:51.539668 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.539521    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-tmp-dir\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:50:51.541238 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.541214    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-dshm" (OuterVolumeSpecName: "dshm") pod "d0df09e2-f325-4947-b093-8788c49c01a6" (UID: "d0df09e2-f325-4947-b093-8788c49c01a6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 04:50:51.541331 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.541251    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0df09e2-f325-4947-b093-8788c49c01a6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0df09e2-f325-4947-b093-8788c49c01a6" (UID: "d0df09e2-f325-4947-b093-8788c49c01a6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 04:50:51.541331 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.541277    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0df09e2-f325-4947-b093-8788c49c01a6-kube-api-access-kjd8l" (OuterVolumeSpecName: "kube-api-access-kjd8l") pod "d0df09e2-f325-4947-b093-8788c49c01a6" (UID: "d0df09e2-f325-4947-b093-8788c49c01a6"). InnerVolumeSpecName "kube-api-access-kjd8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 04:50:51.640011 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.639992    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0df09e2-f325-4947-b093-8788c49c01a6-tls-certs\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:50:51.640011 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.640011    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-dshm\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:50:51.640135 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.640019    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0df09e2-f325-4947-b093-8788c49c01a6-model-cache\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:50:51.640135 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.640028    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjd8l\" (UniqueName: \"kubernetes.io/projected/d0df09e2-f325-4947-b093-8788c49c01a6-kube-api-access-kjd8l\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:50:51.707873 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.707847    2574 generic.go:356] "Generic (PLEG): container finished" podID="d0df09e2-f325-4947-b093-8788c49c01a6" containerID="5a1a9eaceb42c1b9fd555affdf4c2112a4551a6173a74057d4c316d8c614e382" exitCode=0
Jul 28 04:50:51.707961 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.707886    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"
Jul 28 04:50:51.707961 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.707927    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp" event={"ID":"d0df09e2-f325-4947-b093-8788c49c01a6","Type":"ContainerDied","Data":"5a1a9eaceb42c1b9fd555affdf4c2112a4551a6173a74057d4c316d8c614e382"}
Jul 28 04:50:51.708040 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.707966    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp" event={"ID":"d0df09e2-f325-4947-b093-8788c49c01a6","Type":"ContainerDied","Data":"9afcc69473e8a92bd4491296d196ad67c3eff4c804778f31039577f1363b9c24"}
Jul 28 04:50:51.708040 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.707983    2574 scope.go:117] "RemoveContainer" containerID="5a1a9eaceb42c1b9fd555affdf4c2112a4551a6173a74057d4c316d8c614e382"
Jul 28 04:50:51.716386 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.716352    2574 scope.go:117] "RemoveContainer" containerID="5a1a9eaceb42c1b9fd555affdf4c2112a4551a6173a74057d4c316d8c614e382"
Jul 28 04:50:51.716648 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:50:51.716631    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a1a9eaceb42c1b9fd555affdf4c2112a4551a6173a74057d4c316d8c614e382\": container with ID starting with 5a1a9eaceb42c1b9fd555affdf4c2112a4551a6173a74057d4c316d8c614e382 not found: ID does not exist" containerID="5a1a9eaceb42c1b9fd555affdf4c2112a4551a6173a74057d4c316d8c614e382"
Jul 28 04:50:51.716697 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.716657    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a1a9eaceb42c1b9fd555affdf4c2112a4551a6173a74057d4c316d8c614e382"} err="failed to get container status \"5a1a9eaceb42c1b9fd555affdf4c2112a4551a6173a74057d4c316d8c614e382\": rpc error: code = NotFound desc = could not find container \"5a1a9eaceb42c1b9fd555affdf4c2112a4551a6173a74057d4c316d8c614e382\": container with ID starting with 5a1a9eaceb42c1b9fd555affdf4c2112a4551a6173a74057d4c316d8c614e382 not found: ID does not exist"
Jul 28 04:50:51.724098 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.724080    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"]
Jul 28 04:50:51.725605 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:51.725586    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-890677f3/e2e-lifecycle-model-890677f3-kserve-5cf99489f5-njqbp"]
Jul 28 04:50:53.620191 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:50:53.620161    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0df09e2-f325-4947-b093-8788c49c01a6" path="/var/lib/kubelet/pods/d0df09e2-f325-4947-b093-8788c49c01a6/volumes"
Jul 28 04:51:11.816572 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:11.816534    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:12.344091 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:12.344059    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:17.832340 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:17.832305    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:17.897219 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:17.897189    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:18.141724 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:18.141631    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:18.387678 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:18.387640    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:19.479934 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:19.479900    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:19.569406 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:19.569352    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:19.668962 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:19.668918    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:20.456516 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:20.456477    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:20.554092 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:20.554067    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:20.657011 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:20.656985    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:22.083200 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:22.083172    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:51:22.099931 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:22.099901    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:27.726710 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:27.726678    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:27.784837 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:27.784811    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:28.429493 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:28.429465    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:34.190730 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:34.190695    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:51:34.403220 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:34.403193    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:34.513138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:34.513073    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:34.603510 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:34.603484    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:45.157146 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:45.157105    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:45.501996 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:45.501906    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:51:45.684822 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:45.684793    2574 scope.go:117] "RemoveContainer" containerID="fa65563aaee06952ce1b9219df5c4f158cc9351599a12e612a16b061c79fab25"
Jul 28 04:51:45.692621 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:51:45.692604    2574 scope.go:117] "RemoveContainer" containerID="dbcdb563404dfdcec5b35360ddbce50a7901d178403b38d0295edbadd0c5b3ad"
Jul 28 04:52:06.588178 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:06.588148    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:13.291336 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:13.291304    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:13.391232 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:13.391201    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:13.659725 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:13.659659    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:13.975738 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:13.975665    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:14.388152 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:14.388119    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:17.348732 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:17.348698    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:17.429090 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:17.429057    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:17.497069 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:17.497040    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:17.599910 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:17.599841    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:18.011108 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:18.011032    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:18.110660 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:18.110629    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:21.869195 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:21.869166    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:22.233902 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:22.233837    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:22.567544 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:22.567516    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:22.880990 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:22.880914    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:52:23.409165 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:23.409138    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:25.750975 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:25.750947    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:25.852497 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:25.852461    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:26.021939 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:26.021867    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:26.337796 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:26.337761    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:26.642953 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:26.642888    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:26.745328 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:26.745301    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:30.281989 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:30.281956    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:31.683348 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:31.683311    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:52:37.291147 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.291110    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"]
Jul 28 04:52:37.291521 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.291467    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0df09e2-f325-4947-b093-8788c49c01a6" containerName="main"
Jul 28 04:52:37.291521 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.291479    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0df09e2-f325-4947-b093-8788c49c01a6" containerName="main"
Jul 28 04:52:37.291521 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.291500    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ebfe7d1-c1ef-4934-ac25-619d9673e000" containerName="revoke-keys"
Jul 28 04:52:37.291521 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.291505    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ebfe7d1-c1ef-4934-ac25-619d9673e000" containerName="revoke-keys"
Jul 28 04:52:37.291650 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.291561    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ebfe7d1-c1ef-4934-ac25-619d9673e000" containerName="revoke-keys"
Jul 28 04:52:37.291650 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.291571    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0df09e2-f325-4947-b093-8788c49c01a6" containerName="main"
Jul 28 04:52:37.294351 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.294336    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.297213 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.297189    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-20ba8c\"/\"default-dockercfg-vthw2\""
Jul 28 04:52:37.297213 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.297210    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-20ba8c\"/\"openshift-service-ca.crt\""
Jul 28 04:52:37.297425 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.297195    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-20ba8c\"/\"test-model-20ba8c-kserve-self-signed-certs\""
Jul 28 04:52:37.297425 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.297195    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-20ba8c\"/\"kube-root-ca.crt\""
Jul 28 04:52:37.302213 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.302196    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"]
Jul 28 04:52:37.307718 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.307700    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:52:37.389422 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.389394    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-model-cache\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.389547 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.389433    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83803b0f-2b59-447a-b023-ec8a5009851a-tls-certs\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.389547 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.389499    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-tmp-dir\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.389628 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.389543    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-dshm\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.389628 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.389587    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p87h\" (UniqueName: \"kubernetes.io/projected/83803b0f-2b59-447a-b023-ec8a5009851a-kube-api-access-5p87h\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.389628 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.389610    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-home\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.490875 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.490842    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-model-cache\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.491006 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.490890    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83803b0f-2b59-447a-b023-ec8a5009851a-tls-certs\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.491006 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.490928    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-tmp-dir\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.491006 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.490959    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-dshm\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.491181 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.491074    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p87h\" (UniqueName: \"kubernetes.io/projected/83803b0f-2b59-447a-b023-ec8a5009851a-kube-api-access-5p87h\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.491181 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.491104    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-home\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.491452 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.491417    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-model-cache\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.491568 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.491546    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-home\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.491874 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.491850    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-tmp-dir\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.493619 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.493590    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-dshm\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.494083 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.494063    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83803b0f-2b59-447a-b023-ec8a5009851a-tls-certs\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.499102 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.499081    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p87h\" (UniqueName: \"kubernetes.io/projected/83803b0f-2b59-447a-b023-ec8a5009851a-kube-api-access-5p87h\") pod \"test-model-20ba8c-kserve-5d57b45d5c-fv69x\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") " pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.603896 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.603837    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:37.730227 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:37.730069    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"]
Jul 28 04:52:37.733143 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:52:37.733116    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83803b0f_2b59_447a_b023_ec8a5009851a.slice/crio-8db27fe8f4c9052682909cda815e7fb707e07437238fbf537877b701eb565c50 WatchSource:0}: Error finding container 8db27fe8f4c9052682909cda815e7fb707e07437238fbf537877b701eb565c50: Status 404 returned error can't find the container with id 8db27fe8f4c9052682909cda815e7fb707e07437238fbf537877b701eb565c50
Jul 28 04:52:38.217520 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:38.217491    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x" event={"ID":"83803b0f-2b59-447a-b023-ec8a5009851a","Type":"ContainerStarted","Data":"101d24f5e0377d1eb1398f1c88f717e51b37ad361474ff73fa8e049dd10641cb"}
Jul 28 04:52:38.217666 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:38.217527    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x" event={"ID":"83803b0f-2b59-447a-b023-ec8a5009851a","Type":"ContainerStarted","Data":"8db27fe8f4c9052682909cda815e7fb707e07437238fbf537877b701eb565c50"}
Jul 28 04:52:38.232878 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:38.232831    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x" podStartSLOduration=1.232816245 podStartE2EDuration="1.232816245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:52:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:52:38.232068502 +0000 UTC m=+2453.173908221" watchObservedRunningTime="2026-07-28 04:52:38.232816245 +0000 UTC m=+2453.174655963"
Jul 28 04:52:45.388794 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:45.388762    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:52:45.718105 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:45.718042    2574 scope.go:117] "RemoveContainer" containerID="20b5f1712089b1f14f0967b9bebbbec8d8a73d53ed34d09492753a252f0ea09d"
Jul 28 04:52:45.725752 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:45.725735    2574 scope.go:117] "RemoveContainer" containerID="8d004006f06e251cb2a1daf821fc3a1f7e33ec4d3166aa25afa7cdf1799db92a"
Jul 28 04:52:47.604965 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:47.604936    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:47.605450 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:47.604980    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:47.619795 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:47.619775    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:52:48.274846 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:52:48.274823    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:53:04.522031 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:04.521997    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:53:08.875123 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:08.875084    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:53:22.375469 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:22.375438    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:53:39.012978 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:39.012878    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:53:39.111925 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:39.111892    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:53:45.079296 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:45.079265    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"]
Jul 28 04:53:45.097810 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:45.097782    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:53:46.038127 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:46.038095    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:53:46.180131 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:46.180101    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz"]
Jul 28 04:53:46.183406 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:46.183390    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz"
Jul 28 04:53:46.185480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:46.185451    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 04:53:46.185613 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:46.185518    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 04:53:46.191021 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:46.190997    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz"]
Jul 28 04:53:46.325450 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:46.325389    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06a87487-42bd-4d32-94e3-1ad178f01e5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz\" (UID: \"06a87487-42bd-4d32-94e3-1ad178f01e5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz"
Jul 28 04:53:46.426375 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:46.426330    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06a87487-42bd-4d32-94e3-1ad178f01e5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz\" (UID: \"06a87487-42bd-4d32-94e3-1ad178f01e5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz"
Jul 28 04:53:46.426912 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:46.426890    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06a87487-42bd-4d32-94e3-1ad178f01e5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz\" (UID: \"06a87487-42bd-4d32-94e3-1ad178f01e5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz"
Jul 28 04:53:46.495020 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:46.494998    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz"
Jul 28 04:53:46.822822 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:46.822799    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz"]
Jul 28 04:53:46.825310 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:53:46.825286    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a87487_42bd_4d32_94e3_1ad178f01e5c.slice/crio-c889de4ce1c0ecccf396943aa7fcf2df74198a3a886a54b6182db05de23023f5 WatchSource:0}: Error finding container c889de4ce1c0ecccf396943aa7fcf2df74198a3a886a54b6182db05de23023f5: Status 404 returned error can't find the container with id c889de4ce1c0ecccf396943aa7fcf2df74198a3a886a54b6182db05de23023f5
Jul 28 04:53:47.551421 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:47.551393    2574 generic.go:356] "Generic (PLEG): container finished" podID="06a87487-42bd-4d32-94e3-1ad178f01e5c" containerID="187f85ee62b88af5d61ee78cfd61425a18d335a5fa86e3dc9de91b470fbb1e30" exitCode=0
Jul 28 04:53:47.551709 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:47.551482    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz" event={"ID":"06a87487-42bd-4d32-94e3-1ad178f01e5c","Type":"ContainerDied","Data":"187f85ee62b88af5d61ee78cfd61425a18d335a5fa86e3dc9de91b470fbb1e30"}
Jul 28 04:53:47.551709 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:47.551535    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz" event={"ID":"06a87487-42bd-4d32-94e3-1ad178f01e5c","Type":"ContainerStarted","Data":"c889de4ce1c0ecccf396943aa7fcf2df74198a3a886a54b6182db05de23023f5"}
Jul 28 04:53:48.684933 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:48.684909    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz"
Jul 28 04:53:48.846102 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:48.846032    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06a87487-42bd-4d32-94e3-1ad178f01e5c-maas-api-service-ca\") pod \"06a87487-42bd-4d32-94e3-1ad178f01e5c\" (UID: \"06a87487-42bd-4d32-94e3-1ad178f01e5c\") "
Jul 28 04:53:48.846353 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:48.846334    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06a87487-42bd-4d32-94e3-1ad178f01e5c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "06a87487-42bd-4d32-94e3-1ad178f01e5c" (UID: "06a87487-42bd-4d32-94e3-1ad178f01e5c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 04:53:48.946766 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:48.946742    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06a87487-42bd-4d32-94e3-1ad178f01e5c-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:53:49.562163 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:49.562138    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz"
Jul 28 04:53:49.562326 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:49.562180    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz" event={"ID":"06a87487-42bd-4d32-94e3-1ad178f01e5c","Type":"ContainerDied","Data":"c889de4ce1c0ecccf396943aa7fcf2df74198a3a886a54b6182db05de23023f5"}
Jul 28 04:53:49.562326 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:49.562264    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c889de4ce1c0ecccf396943aa7fcf2df74198a3a886a54b6182db05de23023f5"
Jul 28 04:53:49.979769 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:49.979693    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:53:56.248033 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:56.248002    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:53:56.622841 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:56.622813    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:53:56.684838 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:56.684811    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:53:56.786166 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:53:56.786139    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:00.097524 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.097493    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x" podUID="83803b0f-2b59-447a-b023-ec8a5009851a" containerName="main" containerID="cri-o://101d24f5e0377d1eb1398f1c88f717e51b37ad361474ff73fa8e049dd10641cb" gracePeriod=45
Jul 28 04:54:00.343276 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.343253    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:54:00.530836 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.530815    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-dshm\") pod \"83803b0f-2b59-447a-b023-ec8a5009851a\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") "
Jul 28 04:54:00.530968 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.530857    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83803b0f-2b59-447a-b023-ec8a5009851a-tls-certs\") pod \"83803b0f-2b59-447a-b023-ec8a5009851a\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") "
Jul 28 04:54:00.530968 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.530918    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5p87h\" (UniqueName: \"kubernetes.io/projected/83803b0f-2b59-447a-b023-ec8a5009851a-kube-api-access-5p87h\") pod \"83803b0f-2b59-447a-b023-ec8a5009851a\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") "
Jul 28 04:54:00.530968 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.530953    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-model-cache\") pod \"83803b0f-2b59-447a-b023-ec8a5009851a\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") "
Jul 28 04:54:00.531093 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.530975    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-tmp-dir\") pod \"83803b0f-2b59-447a-b023-ec8a5009851a\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") "
Jul 28 04:54:00.531148 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.531095    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-home\") pod \"83803b0f-2b59-447a-b023-ec8a5009851a\" (UID: \"83803b0f-2b59-447a-b023-ec8a5009851a\") "
Jul 28 04:54:00.531226 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.531201    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "83803b0f-2b59-447a-b023-ec8a5009851a" (UID: "83803b0f-2b59-447a-b023-ec8a5009851a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 04:54:00.531279 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.531217    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-model-cache" (OuterVolumeSpecName: "model-cache") pod "83803b0f-2b59-447a-b023-ec8a5009851a" (UID: "83803b0f-2b59-447a-b023-ec8a5009851a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 04:54:00.531394 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.531346    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-home" (OuterVolumeSpecName: "home") pod "83803b0f-2b59-447a-b023-ec8a5009851a" (UID: "83803b0f-2b59-447a-b023-ec8a5009851a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 04:54:00.531511 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.531422    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-model-cache\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:54:00.531511 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.531436    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-tmp-dir\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:54:00.533028 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.533006    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83803b0f-2b59-447a-b023-ec8a5009851a-kube-api-access-5p87h" (OuterVolumeSpecName: "kube-api-access-5p87h") pod "83803b0f-2b59-447a-b023-ec8a5009851a" (UID: "83803b0f-2b59-447a-b023-ec8a5009851a"). InnerVolumeSpecName "kube-api-access-5p87h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 04:54:00.533111 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.533028    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-dshm" (OuterVolumeSpecName: "dshm") pod "83803b0f-2b59-447a-b023-ec8a5009851a" (UID: "83803b0f-2b59-447a-b023-ec8a5009851a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 04:54:00.533111 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.533064    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83803b0f-2b59-447a-b023-ec8a5009851a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "83803b0f-2b59-447a-b023-ec8a5009851a" (UID: "83803b0f-2b59-447a-b023-ec8a5009851a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 04:54:00.619441 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.619418    2574 generic.go:356] "Generic (PLEG): container finished" podID="83803b0f-2b59-447a-b023-ec8a5009851a" containerID="101d24f5e0377d1eb1398f1c88f717e51b37ad361474ff73fa8e049dd10641cb" exitCode=0
Jul 28 04:54:00.619532 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.619451    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"
Jul 28 04:54:00.619532 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.619509    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x" event={"ID":"83803b0f-2b59-447a-b023-ec8a5009851a","Type":"ContainerDied","Data":"101d24f5e0377d1eb1398f1c88f717e51b37ad361474ff73fa8e049dd10641cb"}
Jul 28 04:54:00.619646 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.619552    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x" event={"ID":"83803b0f-2b59-447a-b023-ec8a5009851a","Type":"ContainerDied","Data":"8db27fe8f4c9052682909cda815e7fb707e07437238fbf537877b701eb565c50"}
Jul 28 04:54:00.619646 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.619576    2574 scope.go:117] "RemoveContainer" containerID="101d24f5e0377d1eb1398f1c88f717e51b37ad361474ff73fa8e049dd10641cb"
Jul 28 04:54:00.627608 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.627595    2574 scope.go:117] "RemoveContainer" containerID="101d24f5e0377d1eb1398f1c88f717e51b37ad361474ff73fa8e049dd10641cb"
Jul 28 04:54:00.627840 ip-10-0-133-204 kubenswrapper[2574]: E0728 04:54:00.627818    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"101d24f5e0377d1eb1398f1c88f717e51b37ad361474ff73fa8e049dd10641cb\": container with ID starting with 101d24f5e0377d1eb1398f1c88f717e51b37ad361474ff73fa8e049dd10641cb not found: ID does not exist" containerID="101d24f5e0377d1eb1398f1c88f717e51b37ad361474ff73fa8e049dd10641cb"
Jul 28 04:54:00.627898 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.627850    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"101d24f5e0377d1eb1398f1c88f717e51b37ad361474ff73fa8e049dd10641cb"} err="failed to get container status \"101d24f5e0377d1eb1398f1c88f717e51b37ad361474ff73fa8e049dd10641cb\": rpc error: code = NotFound desc = could not find container \"101d24f5e0377d1eb1398f1c88f717e51b37ad361474ff73fa8e049dd10641cb\": container with ID starting with 101d24f5e0377d1eb1398f1c88f717e51b37ad361474ff73fa8e049dd10641cb not found: ID does not exist"
Jul 28 04:54:00.632617 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.632601    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p87h\" (UniqueName: \"kubernetes.io/projected/83803b0f-2b59-447a-b023-ec8a5009851a-kube-api-access-5p87h\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:54:00.632679 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.632618    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-home\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:54:00.632679 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.632628    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83803b0f-2b59-447a-b023-ec8a5009851a-dshm\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:54:00.632679 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.632635    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83803b0f-2b59-447a-b023-ec8a5009851a-tls-certs\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:54:00.639425 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.638715    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"]
Jul 28 04:54:00.644454 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:00.643210    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-20ba8c/test-model-20ba8c-kserve-5d57b45d5c-fv69x"]
Jul 28 04:54:01.256920 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:01.256887    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:01.269912 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:01.269883    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz"]
Jul 28 04:54:01.272603 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:01.272578    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-20ba8c-zrfzz"]
Jul 28 04:54:01.620678 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:01.620636    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06a87487-42bd-4d32-94e3-1ad178f01e5c" path="/var/lib/kubelet/pods/06a87487-42bd-4d32-94e3-1ad178f01e5c/volumes"
Jul 28 04:54:01.621227 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:01.621208    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83803b0f-2b59-447a-b023-ec8a5009851a" path="/var/lib/kubelet/pods/83803b0f-2b59-447a-b023-ec8a5009851a/volumes"
Jul 28 04:54:01.679161 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:01.679139    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:02.279606 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:02.279577    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:54:07.337097 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:07.337071    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:07.468192 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:07.468159    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:07.617713 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:07.617602    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:07.866135 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:07.866101    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:07.997994 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:07.997924    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb"]
Jul 28 04:54:07.998760 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:07.998740    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06a87487-42bd-4d32-94e3-1ad178f01e5c" containerName="revoke-keys"
Jul 28 04:54:07.998867 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:07.998763    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="06a87487-42bd-4d32-94e3-1ad178f01e5c" containerName="revoke-keys"
Jul 28 04:54:07.998867 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:07.998778    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83803b0f-2b59-447a-b023-ec8a5009851a" containerName="main"
Jul 28 04:54:07.998867 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:07.998794    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="83803b0f-2b59-447a-b023-ec8a5009851a" containerName="main"
Jul 28 04:54:07.999001 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:07.998959    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="06a87487-42bd-4d32-94e3-1ad178f01e5c" containerName="revoke-keys"
Jul 28 04:54:07.999001 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:07.998976    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="83803b0f-2b59-447a-b023-ec8a5009851a" containerName="main"
Jul 28 04:54:08.004000 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:08.003981    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb"
Jul 28 04:54:08.006083 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:08.006051    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 04:54:08.006498 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:08.006458    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 04:54:08.007120 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:08.007099    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb"]
Jul 28 04:54:08.086971 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:08.086944    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/767daa18-61ee-4330-a92e-a770155dc0b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb\" (UID: \"767daa18-61ee-4330-a92e-a770155dc0b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb"
Jul 28 04:54:08.187763 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:08.187741    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/767daa18-61ee-4330-a92e-a770155dc0b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb\" (UID: \"767daa18-61ee-4330-a92e-a770155dc0b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb"
Jul 28 04:54:08.188287 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:08.188269    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/767daa18-61ee-4330-a92e-a770155dc0b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb\" (UID: \"767daa18-61ee-4330-a92e-a770155dc0b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb"
Jul 28 04:54:08.317592 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:08.317568    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb"
Jul 28 04:54:08.441489 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:08.441464    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb"]
Jul 28 04:54:08.446635 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:54:08.446604    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod767daa18_61ee_4330_a92e_a770155dc0b8.slice/crio-bfcdd602bf0629928eed1d1753a5f28a226d530808bba2c48eb223f099c2f7bf WatchSource:0}: Error finding container bfcdd602bf0629928eed1d1753a5f28a226d530808bba2c48eb223f099c2f7bf: Status 404 returned error can't find the container with id bfcdd602bf0629928eed1d1753a5f28a226d530808bba2c48eb223f099c2f7bf
Jul 28 04:54:08.656261 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:08.656234    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb" event={"ID":"767daa18-61ee-4330-a92e-a770155dc0b8","Type":"ContainerStarted","Data":"bfcdd602bf0629928eed1d1753a5f28a226d530808bba2c48eb223f099c2f7bf"}
Jul 28 04:54:09.661785 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:09.661758    2574 generic.go:356] "Generic (PLEG): container finished" podID="767daa18-61ee-4330-a92e-a770155dc0b8" containerID="78bbfa991a788504efcc3b2d4fd7c9dd0f64555291bafdfa2299b57a95af6b4d" exitCode=0
Jul 28 04:54:09.662130 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:09.661848    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb" event={"ID":"767daa18-61ee-4330-a92e-a770155dc0b8","Type":"ContainerDied","Data":"78bbfa991a788504efcc3b2d4fd7c9dd0f64555291bafdfa2299b57a95af6b4d"}
Jul 28 04:54:10.793558 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:10.793537    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb"
Jul 28 04:54:10.807776 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:10.807751    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/767daa18-61ee-4330-a92e-a770155dc0b8-maas-api-service-ca\") pod \"767daa18-61ee-4330-a92e-a770155dc0b8\" (UID: \"767daa18-61ee-4330-a92e-a770155dc0b8\") "
Jul 28 04:54:10.808205 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:10.808182    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/767daa18-61ee-4330-a92e-a770155dc0b8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "767daa18-61ee-4330-a92e-a770155dc0b8" (UID: "767daa18-61ee-4330-a92e-a770155dc0b8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 04:54:10.908645 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:10.908621    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/767daa18-61ee-4330-a92e-a770155dc0b8-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:54:11.671318 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:11.671293    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb"
Jul 28 04:54:11.671442 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:11.671319    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb" event={"ID":"767daa18-61ee-4330-a92e-a770155dc0b8","Type":"ContainerDied","Data":"bfcdd602bf0629928eed1d1753a5f28a226d530808bba2c48eb223f099c2f7bf"}
Jul 28 04:54:11.671442 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:11.671369    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bfcdd602bf0629928eed1d1753a5f28a226d530808bba2c48eb223f099c2f7bf"
Jul 28 04:54:18.448690 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:18.448643    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:18.546675 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:18.546639    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:18.647009 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:18.646978    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:21.380332 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:21.380297    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:54:23.077134 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:23.077103    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb"]
Jul 28 04:54:23.081506 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:23.081478    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b1e7a-cwtnb"]
Jul 28 04:54:23.081624 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:23.081583    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:23.501050 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:23.500960    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:23.620144 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:23.620120    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="767daa18-61ee-4330-a92e-a770155dc0b8" path="/var/lib/kubelet/pods/767daa18-61ee-4330-a92e-a770155dc0b8/volumes"
Jul 28 04:54:29.245766 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:29.245731    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:29.305142 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:29.305116    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:29.532142 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:29.532112    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:29.788208 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:29.788136    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:30.160879 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:30.160804    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:30.513017 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:30.512953    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:30.613029 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:30.612999    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:33.052195 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:33.052163    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:33.150984 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:33.150955    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:33.314683 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:33.314612    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:33.557249 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:33.557215    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:33.782534 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:33.782510    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:33.883490 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:33.883450    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:36.087277 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:36.087248    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:54:37.621278 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:37.621248    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:38.140570 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:38.140542    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:38.472253 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:38.472184    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:39.264537 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:39.264507    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:40.418346 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:40.418317    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:41.979156 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:41.979119    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:42.056034 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:42.056006    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:45.696138 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:45.696105    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:54:45.754704 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:45.754682    2574 scope.go:117] "RemoveContainer" containerID="8e3e9efdee8edda7ca739b1b026309b2ed7acd971c4d667694a74d23a8313537"
Jul 28 04:54:48.583220 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:48.583190    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:54:53.959626 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:53.959596    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v"]
Jul 28 04:54:53.960061 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:53.959933    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="767daa18-61ee-4330-a92e-a770155dc0b8" containerName="revoke-keys"
Jul 28 04:54:53.960061 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:53.959943    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="767daa18-61ee-4330-a92e-a770155dc0b8" containerName="revoke-keys"
Jul 28 04:54:53.960061 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:53.960007    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="767daa18-61ee-4330-a92e-a770155dc0b8" containerName="revoke-keys"
Jul 28 04:54:53.962706 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:53.962691    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v"
Jul 28 04:54:53.964518 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:53.964490    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 04:54:53.964639 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:53.964607    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 04:54:53.968617 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:53.968598    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v"]
Jul 28 04:54:54.012627 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:54.012605    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d4a3929-252b-4317-b7fc-336634def88f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f397e0-rp52v\" (UID: \"9d4a3929-252b-4317-b7fc-336634def88f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v"
Jul 28 04:54:54.113927 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:54.113901    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d4a3929-252b-4317-b7fc-336634def88f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f397e0-rp52v\" (UID: \"9d4a3929-252b-4317-b7fc-336634def88f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v"
Jul 28 04:54:54.114483 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:54.114467    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d4a3929-252b-4317-b7fc-336634def88f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f397e0-rp52v\" (UID: \"9d4a3929-252b-4317-b7fc-336634def88f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v"
Jul 28 04:54:54.274201 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:54.274142    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v"
Jul 28 04:54:54.395371 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:54.395338    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v"]
Jul 28 04:54:54.397934 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:54:54.397902    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d4a3929_252b_4317_b7fc_336634def88f.slice/crio-344a9fc5f4b8799e24029a95208de8a3fd7d8b6feb29a1f9cb8413cd2d01cdc4 WatchSource:0}: Error finding container 344a9fc5f4b8799e24029a95208de8a3fd7d8b6feb29a1f9cb8413cd2d01cdc4: Status 404 returned error can't find the container with id 344a9fc5f4b8799e24029a95208de8a3fd7d8b6feb29a1f9cb8413cd2d01cdc4
Jul 28 04:54:54.869443 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:54.869350    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v" event={"ID":"9d4a3929-252b-4317-b7fc-336634def88f","Type":"ContainerStarted","Data":"26e46f5eed68aef56996ec745b66c048a451ee824855d30e70250f2c2610808d"}
Jul 28 04:54:54.869443 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:54.869417    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v" event={"ID":"9d4a3929-252b-4317-b7fc-336634def88f","Type":"ContainerStarted","Data":"344a9fc5f4b8799e24029a95208de8a3fd7d8b6feb29a1f9cb8413cd2d01cdc4"}
Jul 28 04:54:54.884108 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:54.884061    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v" podStartSLOduration=1.884046091 podStartE2EDuration="1.884046091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:54:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:54:54.881926369 +0000 UTC m=+2589.823766089" watchObservedRunningTime="2026-07-28 04:54:54.884046091 +0000 UTC m=+2589.825885790"
Jul 28 04:54:55.874639 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:55.874605    2574 generic.go:356] "Generic (PLEG): container finished" podID="9d4a3929-252b-4317-b7fc-336634def88f" containerID="26e46f5eed68aef56996ec745b66c048a451ee824855d30e70250f2c2610808d" exitCode=0
Jul 28 04:54:55.875006 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:55.874686    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v" event={"ID":"9d4a3929-252b-4317-b7fc-336634def88f","Type":"ContainerDied","Data":"26e46f5eed68aef56996ec745b66c048a451ee824855d30e70250f2c2610808d"}
Jul 28 04:54:57.018582 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:57.018560    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v"
Jul 28 04:54:57.135988 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:57.135932    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d4a3929-252b-4317-b7fc-336634def88f-maas-api-service-ca\") pod \"9d4a3929-252b-4317-b7fc-336634def88f\" (UID: \"9d4a3929-252b-4317-b7fc-336634def88f\") "
Jul 28 04:54:57.136237 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:57.136218    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d4a3929-252b-4317-b7fc-336634def88f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9d4a3929-252b-4317-b7fc-336634def88f" (UID: "9d4a3929-252b-4317-b7fc-336634def88f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 04:54:57.236789 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:57.236766    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d4a3929-252b-4317-b7fc-336634def88f-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:54:57.891340 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:57.891275    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v"
Jul 28 04:54:57.891340 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:57.891317    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v" event={"ID":"9d4a3929-252b-4317-b7fc-336634def88f","Type":"ContainerDied","Data":"344a9fc5f4b8799e24029a95208de8a3fd7d8b6feb29a1f9cb8413cd2d01cdc4"}
Jul 28 04:54:57.891522 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:54:57.891404    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="344a9fc5f4b8799e24029a95208de8a3fd7d8b6feb29a1f9cb8413cd2d01cdc4"
Jul 28 04:55:09.062068 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:09.061991    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:09.162989 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:09.162960    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:09.224839 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:09.224798    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:11.883262 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:11.879853    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:55:14.040014 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:14.039978    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v"]
Jul 28 04:55:14.042332 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:14.042307    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f397e0-rp52v"]
Jul 28 04:55:14.042480 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:14.042379    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:14.457819 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:14.457738    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:15.620236 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:15.620209    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d4a3929-252b-4317-b7fc-336634def88f" path="/var/lib/kubelet/pods/9d4a3929-252b-4317-b7fc-336634def88f/volumes"
Jul 28 04:55:20.168168 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:20.168122    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:20.224907 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:20.224883    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:20.442560 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:20.442482    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:20.705713 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:20.705632    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:20.823374 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:20.823338    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9"]
Jul 28 04:55:20.823704 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:20.823691    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d4a3929-252b-4317-b7fc-336634def88f" containerName="revoke-keys"
Jul 28 04:55:20.823746 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:20.823705    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d4a3929-252b-4317-b7fc-336634def88f" containerName="revoke-keys"
Jul 28 04:55:20.823779 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:20.823774    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d4a3929-252b-4317-b7fc-336634def88f" containerName="revoke-keys"
Jul 28 04:55:20.826909 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:20.826895    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9"
Jul 28 04:55:20.829122 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:20.829099    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 04:55:20.829233 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:20.829102    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 04:55:20.832258 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:20.832236    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9"]
Jul 28 04:55:20.906521 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:20.906500    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9fd2ff4-0681-4b8e-b91d-490094abe459-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9\" (UID: \"d9fd2ff4-0681-4b8e-b91d-490094abe459\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9"
Jul 28 04:55:21.006926 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:21.006875    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9fd2ff4-0681-4b8e-b91d-490094abe459-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9\" (UID: \"d9fd2ff4-0681-4b8e-b91d-490094abe459\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9"
Jul 28 04:55:21.007431 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:21.007414    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9fd2ff4-0681-4b8e-b91d-490094abe459-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9\" (UID: \"d9fd2ff4-0681-4b8e-b91d-490094abe459\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9"
Jul 28 04:55:21.137815 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:21.137794    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9"
Jul 28 04:55:21.465608 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:21.465586    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9"]
Jul 28 04:55:21.467659 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:55:21.467631    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9fd2ff4_0681_4b8e_b91d_490094abe459.slice/crio-ed28f431838840fa4920e86d3d3df998ccd9b090a62a4c1d7153bee38b2f262d WatchSource:0}: Error finding container ed28f431838840fa4920e86d3d3df998ccd9b090a62a4c1d7153bee38b2f262d: Status 404 returned error can't find the container with id ed28f431838840fa4920e86d3d3df998ccd9b090a62a4c1d7153bee38b2f262d
Jul 28 04:55:22.001219 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:22.001192    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9" event={"ID":"d9fd2ff4-0681-4b8e-b91d-490094abe459","Type":"ContainerStarted","Data":"fa2b8ae14d10d034ae1f215ed0179f24d38b4005c0544ee30e2fac04c950d5a9"}
Jul 28 04:55:22.001354 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:22.001237    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9" event={"ID":"d9fd2ff4-0681-4b8e-b91d-490094abe459","Type":"ContainerStarted","Data":"ed28f431838840fa4920e86d3d3df998ccd9b090a62a4c1d7153bee38b2f262d"}
Jul 28 04:55:22.015503 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:22.015460    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9" podStartSLOduration=2.015443271 podStartE2EDuration="2.015443271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:55:22.0141657 +0000 UTC m=+2616.956005417" watchObservedRunningTime="2026-07-28 04:55:22.015443271 +0000 UTC m=+2616.957283020"
Jul 28 04:55:23.007899 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:23.007866    2574 generic.go:356] "Generic (PLEG): container finished" podID="d9fd2ff4-0681-4b8e-b91d-490094abe459" containerID="fa2b8ae14d10d034ae1f215ed0179f24d38b4005c0544ee30e2fac04c950d5a9" exitCode=0
Jul 28 04:55:23.008252 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:23.007943    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9" event={"ID":"d9fd2ff4-0681-4b8e-b91d-490094abe459","Type":"ContainerDied","Data":"fa2b8ae14d10d034ae1f215ed0179f24d38b4005c0544ee30e2fac04c950d5a9"}
Jul 28 04:55:24.139664 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:24.139643    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9"
Jul 28 04:55:24.232088 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:24.232068    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9fd2ff4-0681-4b8e-b91d-490094abe459-maas-api-service-ca\") pod \"d9fd2ff4-0681-4b8e-b91d-490094abe459\" (UID: \"d9fd2ff4-0681-4b8e-b91d-490094abe459\") "
Jul 28 04:55:24.232423 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:24.232403    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9fd2ff4-0681-4b8e-b91d-490094abe459-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d9fd2ff4-0681-4b8e-b91d-490094abe459" (UID: "d9fd2ff4-0681-4b8e-b91d-490094abe459"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 04:55:24.333449 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:24.333426    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9fd2ff4-0681-4b8e-b91d-490094abe459-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 04:55:25.018282 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:25.018257    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9"
Jul 28 04:55:25.018550 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:25.018303    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9" event={"ID":"d9fd2ff4-0681-4b8e-b91d-490094abe459","Type":"ContainerDied","Data":"ed28f431838840fa4920e86d3d3df998ccd9b090a62a4c1d7153bee38b2f262d"}
Jul 28 04:55:25.018550 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:25.018344    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed28f431838840fa4920e86d3d3df998ccd9b090a62a4c1d7153bee38b2f262d"
Jul 28 04:55:33.844382 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:33.844333    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:33.896990 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:33.896961    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:36.291911 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:36.291878    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:36.360540 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:36.360513    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:36.418703 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:36.418680    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:40.907992 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:40.907958    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:40.908842 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:40.908822    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9"]
Jul 28 04:55:40.911021 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:40.910999    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1b2859-hcjf9"]
Jul 28 04:55:41.319043 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:41.319012    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:41.379586 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:41.379558    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:55:41.620154 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:41.620089    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9fd2ff4-0681-4b8e-b91d-490094abe459" path="/var/lib/kubelet/pods/d9fd2ff4-0681-4b8e-b91d-490094abe459/volumes"
Jul 28 04:55:47.021485 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:47.021453    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:47.091926 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:47.091899    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:47.385955 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:47.385921    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:47.576447 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:47.576420    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:47.944536 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:47.944508    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:48.269303 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:48.269237    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:48.456319 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:48.456287    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:51.517259 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:51.517226    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:51.617376 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:51.617338    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:51.788668 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:51.788643    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:51.968789 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:51.968759    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:52.323048 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:52.323023    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:52.421681 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:52.421653    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:55.871657 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:55.871626    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:55:56.041006 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:56.040974    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:56.413749 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:56.413726    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:56.720675 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:56.720606    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:57.384323 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:57.384291    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:59.122174 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:59.122136    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:55:59.222588 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:55:59.222557    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:56:00.056152 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:00.056112    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:56:00.163349 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:00.163323    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:56:00.255047 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:00.255020    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:56:00.358055 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:00.357987    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:56:00.852576 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:00.852528    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:56:00.956150 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:00.956117    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:56:04.537518 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:04.537467    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:56:08.480856 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:08.480821    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:56:08.731680 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:08.731611    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:56:12.910289 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:12.910257    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:56:18.877885 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:18.877854    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:56:19.660924 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:19.660893    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:56:45.786229 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:45.786154    2574 scope.go:117] "RemoveContainer" containerID="f461556ba1223fd5f282dfefc5a2728ca251b5a7dfab20817c84c45f11df56d3"
Jul 28 04:56:51.600626 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.600590    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"]
Jul 28 04:56:51.600997 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.600984    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9fd2ff4-0681-4b8e-b91d-490094abe459" containerName="revoke-keys"
Jul 28 04:56:51.600997 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.600998    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9fd2ff4-0681-4b8e-b91d-490094abe459" containerName="revoke-keys"
Jul 28 04:56:51.601070 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.601058    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9fd2ff4-0681-4b8e-b91d-490094abe459" containerName="revoke-keys"
Jul 28 04:56:51.605566 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.605549    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.607530 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.607510    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-92c557\"/\"default-dockercfg-gttgr\""
Jul 28 04:56:51.608157 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.608140    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-92c557\"/\"auth-test-model-92c557-kserve-self-signed-certs\""
Jul 28 04:56:51.608201 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.608167    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-92c557\"/\"openshift-service-ca.crt\""
Jul 28 04:56:51.608243 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.608149    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-92c557\"/\"kube-root-ca.crt\""
Jul 28 04:56:51.612065 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.612042    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"]
Jul 28 04:56:51.617881 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.617857    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:56:51.660402 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.660343    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-tmp-dir\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.660541 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.660464    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-dshm\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.660541 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.660488    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-model-cache\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.660662 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.660600    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/188055e8-41f9-4acf-86de-08f713e4398f-tls-certs\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.660662 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.660641    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv85p\" (UniqueName: \"kubernetes.io/projected/188055e8-41f9-4acf-86de-08f713e4398f-kube-api-access-mv85p\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.660749 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.660707    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-home\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.761397 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.761341    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-dshm\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.761397 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.761396    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-model-cache\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.761614 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.761436    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/188055e8-41f9-4acf-86de-08f713e4398f-tls-certs\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.761614 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.761455    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv85p\" (UniqueName: \"kubernetes.io/projected/188055e8-41f9-4acf-86de-08f713e4398f-kube-api-access-mv85p\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.761614 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.761490    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-home\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.761773 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.761631    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-tmp-dir\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.761832 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.761795    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-home\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.761886 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.761840    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-model-cache\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.761965 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.761920    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-tmp-dir\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.763594 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.763572    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-dshm\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.763876 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.763856    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/188055e8-41f9-4acf-86de-08f713e4398f-tls-certs\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.768339 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.768321    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv85p\" (UniqueName: \"kubernetes.io/projected/188055e8-41f9-4acf-86de-08f713e4398f-kube-api-access-mv85p\") pod \"auth-test-model-92c557-kserve-64d758c9b7-c9msm\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") " pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:51.914917 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:51.914818    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:56:52.044780 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:52.044748    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"]
Jul 28 04:56:52.047708 ip-10-0-133-204 kubenswrapper[2574]: W0728 04:56:52.047680    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod188055e8_41f9_4acf_86de_08f713e4398f.slice/crio-7d323592988952821e8b3a9f1bf5a6a2f625ff2d304b834a431d7e0ee69b5bd4 WatchSource:0}: Error finding container 7d323592988952821e8b3a9f1bf5a6a2f625ff2d304b834a431d7e0ee69b5bd4: Status 404 returned error can't find the container with id 7d323592988952821e8b3a9f1bf5a6a2f625ff2d304b834a431d7e0ee69b5bd4
Jul 28 04:56:52.437178 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:52.437142    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm" event={"ID":"188055e8-41f9-4acf-86de-08f713e4398f","Type":"ContainerStarted","Data":"2ca91e0394935ea7bebfe39edec3b6c624cddbecf122888afbc4b9e4bc2ac61b"}
Jul 28 04:56:52.437387 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:52.437191    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm" event={"ID":"188055e8-41f9-4acf-86de-08f713e4398f","Type":"ContainerStarted","Data":"7d323592988952821e8b3a9f1bf5a6a2f625ff2d304b834a431d7e0ee69b5bd4"}
Jul 28 04:56:52.455216 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:52.455170    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm" podStartSLOduration=1.455155143 podStartE2EDuration="1.455155143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 04:56:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 04:56:52.453960704 +0000 UTC m=+2707.395800434" watchObservedRunningTime="2026-07-28 04:56:52.455155143 +0000 UTC m=+2707.396994861"
Jul 28 04:56:54.903546 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:56:54.903514    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:57:01.915079 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:57:01.915044    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:57:01.915079 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:57:01.915089    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:57:01.927081 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:57:01.927058    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:57:02.491080 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:57:02.491055    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 04:57:26.676010 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:57:26.675976    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:57:47.374879 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:57:47.374845    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:58:02.882646 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:58:02.882609    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:58:09.607991 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:58:09.607912    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:58:26.073805 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:58:26.073771    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:58:43.186541 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:58:43.186508    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:58:51.334813 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:58:51.334746    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 04:59:03.884024 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:59:03.883977    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:59:20.884167 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:59:20.884133    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 04:59:42.983004 ip-10-0-133-204 kubenswrapper[2574]: I0728 04:59:42.982932    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:00:00.154569 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.154532    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf"]
Jul 28 05:00:00.157681 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.157658    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v"]
Jul 28 05:00:00.157817 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.157742    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf"
Jul 28 05:00:00.160434 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.160408    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 05:00:00.160777 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.160758    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753580-tq6qw"]
Jul 28 05:00:00.160896 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.160880    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v"
Jul 28 05:00:00.163492 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.163474    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753580-tq6qw"
Jul 28 05:00:00.167967 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.167948    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf"
Jul 28 05:00:00.177702 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.177682    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v"
Jul 28 05:00:00.198123 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.198091    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v"]
Jul 28 05:00:00.209945 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.209924    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753580-tq6qw"
Jul 28 05:00:00.233447 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.233418    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf"]
Jul 28 05:00:00.288701 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.288634    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753580-tq6qw"]
Jul 28 05:00:00.514343 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.514316    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf"]
Jul 28 05:00:00.518964 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:00:00.518933    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce95dfe8_f309_44b8_987e_454890e68b7b.slice/crio-b219368ddc8c81e4a275895ccd59b9134e84cf2ca1e0b82971b417ce67363323 WatchSource:0}: Error finding container b219368ddc8c81e4a275895ccd59b9134e84cf2ca1e0b82971b417ce67363323: Status 404 returned error can't find the container with id b219368ddc8c81e4a275895ccd59b9134e84cf2ca1e0b82971b417ce67363323
Jul 28 05:00:00.532736 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.532703    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v"]
Jul 28 05:00:00.536355 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:00:00.536320    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7063fed_a260_4312_860d_54529267118b.slice/crio-00a0f1ba3fb9d7be44f2a8c88f7520f9a37603c11485a99c377aa648892f0a68 WatchSource:0}: Error finding container 00a0f1ba3fb9d7be44f2a8c88f7520f9a37603c11485a99c377aa648892f0a68: Status 404 returned error can't find the container with id 00a0f1ba3fb9d7be44f2a8c88f7520f9a37603c11485a99c377aa648892f0a68
Jul 28 05:00:00.567590 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:00.567564    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753580-tq6qw"]
Jul 28 05:00:00.570239 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:00:00.570213    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f427be_6c0a_4363_9332_5bca149df170.slice/crio-2ebee55c40c7e9864715707f32f22f8fa8301b6c85a080e459d3cf53a5ea9100 WatchSource:0}: Error finding container 2ebee55c40c7e9864715707f32f22f8fa8301b6c85a080e459d3cf53a5ea9100: Status 404 returned error can't find the container with id 2ebee55c40c7e9864715707f32f22f8fa8301b6c85a080e459d3cf53a5ea9100
Jul 28 05:00:01.336577 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:01.336264    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf" event={"ID":"ce95dfe8-f309-44b8-987e-454890e68b7b","Type":"ContainerStarted","Data":"0687f09bd0d222a1b2f9b805b968b267ed8b54ca244ef4cd2ea43634addd4a7e"}
Jul 28 05:00:01.336577 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:01.336393    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf" event={"ID":"ce95dfe8-f309-44b8-987e-454890e68b7b","Type":"ContainerStarted","Data":"b219368ddc8c81e4a275895ccd59b9134e84cf2ca1e0b82971b417ce67363323"}
Jul 28 05:00:01.338081 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:01.338053    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753580-tq6qw" event={"ID":"03f427be-6c0a-4363-9332-5bca149df170","Type":"ContainerStarted","Data":"75e6c71e7a4d48a6c9e83f78395e638525655dd4bf5b0df61ec7b2da7fc42cc2"}
Jul 28 05:00:01.338215 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:01.338104    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753580-tq6qw" event={"ID":"03f427be-6c0a-4363-9332-5bca149df170","Type":"ContainerStarted","Data":"2ebee55c40c7e9864715707f32f22f8fa8301b6c85a080e459d3cf53a5ea9100"}
Jul 28 05:00:01.339587 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:01.339565    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v" event={"ID":"c7063fed-a260-4312-860d-54529267118b","Type":"ContainerStarted","Data":"ddb1e5fb669bfc8779d344055846bed188c8a43463de4b41b71a213c5ceb3be4"}
Jul 28 05:00:01.339699 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:01.339598    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v" event={"ID":"c7063fed-a260-4312-860d-54529267118b","Type":"ContainerStarted","Data":"00a0f1ba3fb9d7be44f2a8c88f7520f9a37603c11485a99c377aa648892f0a68"}
Jul 28 05:00:01.350309 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:01.350258    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf" podStartSLOduration=1.350242697 podStartE2EDuration="1.350242697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 05:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 05:00:01.34850362 +0000 UTC m=+2896.290343347" watchObservedRunningTime="2026-07-28 05:00:01.350242697 +0000 UTC m=+2896.292082417"
Jul 28 05:00:01.361617 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:01.361549    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753580-tq6qw" podStartSLOduration=1.361530672 podStartE2EDuration="1.361530672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 05:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 05:00:01.361175258 +0000 UTC m=+2896.303014978" watchObservedRunningTime="2026-07-28 05:00:01.361530672 +0000 UTC m=+2896.303370394"
Jul 28 05:00:01.374673 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:01.374622    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v" podStartSLOduration=1.374606467 podStartE2EDuration="1.374606467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 05:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 05:00:01.373192144 +0000 UTC m=+2896.315031877" watchObservedRunningTime="2026-07-28 05:00:01.374606467 +0000 UTC m=+2896.316446186"
Jul 28 05:00:01.378657 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:01.378629    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:00:02.345995 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:02.345956    2574 generic.go:356] "Generic (PLEG): container finished" podID="c7063fed-a260-4312-860d-54529267118b" containerID="ddb1e5fb669bfc8779d344055846bed188c8a43463de4b41b71a213c5ceb3be4" exitCode=0
Jul 28 05:00:02.346431 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:02.346042    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v" event={"ID":"c7063fed-a260-4312-860d-54529267118b","Type":"ContainerDied","Data":"ddb1e5fb669bfc8779d344055846bed188c8a43463de4b41b71a213c5ceb3be4"}
Jul 28 05:00:02.347452 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:02.347429    2574 generic.go:356] "Generic (PLEG): container finished" podID="ce95dfe8-f309-44b8-987e-454890e68b7b" containerID="0687f09bd0d222a1b2f9b805b968b267ed8b54ca244ef4cd2ea43634addd4a7e" exitCode=0
Jul 28 05:00:02.347560 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:02.347454    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf" event={"ID":"ce95dfe8-f309-44b8-987e-454890e68b7b","Type":"ContainerDied","Data":"0687f09bd0d222a1b2f9b805b968b267ed8b54ca244ef4cd2ea43634addd4a7e"}
Jul 28 05:00:02.349105 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:02.349083    2574 generic.go:356] "Generic (PLEG): container finished" podID="03f427be-6c0a-4363-9332-5bca149df170" containerID="75e6c71e7a4d48a6c9e83f78395e638525655dd4bf5b0df61ec7b2da7fc42cc2" exitCode=0
Jul 28 05:00:02.349187 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:02.349143    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753580-tq6qw" event={"ID":"03f427be-6c0a-4363-9332-5bca149df170","Type":"ContainerDied","Data":"75e6c71e7a4d48a6c9e83f78395e638525655dd4bf5b0df61ec7b2da7fc42cc2"}
Jul 28 05:00:03.505801 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:03.505777    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v"
Jul 28 05:00:03.555911 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:03.555890    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753580-tq6qw"
Jul 28 05:00:03.558793 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:03.558773    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf"
Jul 28 05:00:04.361045 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:04.361013    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf"
Jul 28 05:00:04.361230 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:04.361016    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf" event={"ID":"ce95dfe8-f309-44b8-987e-454890e68b7b","Type":"ContainerDied","Data":"b219368ddc8c81e4a275895ccd59b9134e84cf2ca1e0b82971b417ce67363323"}
Jul 28 05:00:04.361230 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:04.361126    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b219368ddc8c81e4a275895ccd59b9134e84cf2ca1e0b82971b417ce67363323"
Jul 28 05:00:04.362345 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:04.362321    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753580-tq6qw"
Jul 28 05:00:04.362470 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:04.362372    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753580-tq6qw" event={"ID":"03f427be-6c0a-4363-9332-5bca149df170","Type":"ContainerDied","Data":"2ebee55c40c7e9864715707f32f22f8fa8301b6c85a080e459d3cf53a5ea9100"}
Jul 28 05:00:04.362470 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:04.362417    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ebee55c40c7e9864715707f32f22f8fa8301b6c85a080e459d3cf53a5ea9100"
Jul 28 05:00:04.363801 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:04.363780    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v" event={"ID":"c7063fed-a260-4312-860d-54529267118b","Type":"ContainerDied","Data":"00a0f1ba3fb9d7be44f2a8c88f7520f9a37603c11485a99c377aa648892f0a68"}
Jul 28 05:00:04.363911 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:04.363802    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v"
Jul 28 05:00:04.363911 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:04.363816    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00a0f1ba3fb9d7be44f2a8c88f7520f9a37603c11485a99c377aa648892f0a68"
Jul 28 05:00:24.874790 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:24.874755    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:00:27.559614 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:27.559581    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:00:28.927007 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:28.926974    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:00:40.973018 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:40.972984    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:00:45.825484 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:45.825449    2574 scope.go:117] "RemoveContainer" containerID="187f85ee62b88af5d61ee78cfd61425a18d335a5fa86e3dc9de91b470fbb1e30"
Jul 28 05:00:45.833939 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:00:45.833920    2574 scope.go:117] "RemoveContainer" containerID="78bbfa991a788504efcc3b2d4fd7c9dd0f64555291bafdfa2299b57a95af6b4d"
Jul 28 05:01:01.676838 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:01.676804    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:01:06.960535 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:06.960500    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:01:07.794099 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:07.794016    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"]
Jul 28 05:01:07.812075 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:07.812049    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:01:18.979668 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:18.979628    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:01:22.813243 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:22.813181    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm" podUID="188055e8-41f9-4acf-86de-08f713e4398f" containerName="main" containerID="cri-o://2ca91e0394935ea7bebfe39edec3b6c624cddbecf122888afbc4b9e4bc2ac61b" gracePeriod=45
Jul 28 05:01:23.078066 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.077995    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 05:01:23.110843 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.110813    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-model-cache\") pod \"188055e8-41f9-4acf-86de-08f713e4398f\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") "
Jul 28 05:01:23.111021 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.110867    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-dshm\") pod \"188055e8-41f9-4acf-86de-08f713e4398f\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") "
Jul 28 05:01:23.111021 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.110883    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-home\") pod \"188055e8-41f9-4acf-86de-08f713e4398f\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") "
Jul 28 05:01:23.111021 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.110918    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mv85p\" (UniqueName: \"kubernetes.io/projected/188055e8-41f9-4acf-86de-08f713e4398f-kube-api-access-mv85p\") pod \"188055e8-41f9-4acf-86de-08f713e4398f\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") "
Jul 28 05:01:23.111021 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.110959    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/188055e8-41f9-4acf-86de-08f713e4398f-tls-certs\") pod \"188055e8-41f9-4acf-86de-08f713e4398f\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") "
Jul 28 05:01:23.111241 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.111086    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-tmp-dir\") pod \"188055e8-41f9-4acf-86de-08f713e4398f\" (UID: \"188055e8-41f9-4acf-86de-08f713e4398f\") "
Jul 28 05:01:23.111241 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.111129    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-model-cache" (OuterVolumeSpecName: "model-cache") pod "188055e8-41f9-4acf-86de-08f713e4398f" (UID: "188055e8-41f9-4acf-86de-08f713e4398f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:01:23.111330 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.111239    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-home" (OuterVolumeSpecName: "home") pod "188055e8-41f9-4acf-86de-08f713e4398f" (UID: "188055e8-41f9-4acf-86de-08f713e4398f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:01:23.111418 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.111385    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "188055e8-41f9-4acf-86de-08f713e4398f" (UID: "188055e8-41f9-4acf-86de-08f713e4398f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:01:23.111471 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.111455    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-model-cache\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:01:23.111526 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.111477    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-home\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:01:23.113119 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.113090    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-dshm" (OuterVolumeSpecName: "dshm") pod "188055e8-41f9-4acf-86de-08f713e4398f" (UID: "188055e8-41f9-4acf-86de-08f713e4398f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:01:23.113119 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.113114    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/188055e8-41f9-4acf-86de-08f713e4398f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "188055e8-41f9-4acf-86de-08f713e4398f" (UID: "188055e8-41f9-4acf-86de-08f713e4398f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 05:01:23.113268 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.113209    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/188055e8-41f9-4acf-86de-08f713e4398f-kube-api-access-mv85p" (OuterVolumeSpecName: "kube-api-access-mv85p") pod "188055e8-41f9-4acf-86de-08f713e4398f" (UID: "188055e8-41f9-4acf-86de-08f713e4398f"). InnerVolumeSpecName "kube-api-access-mv85p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 05:01:23.212314 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.212274    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-tmp-dir\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:01:23.212314 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.212307    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/188055e8-41f9-4acf-86de-08f713e4398f-dshm\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:01:23.212314 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.212317    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv85p\" (UniqueName: \"kubernetes.io/projected/188055e8-41f9-4acf-86de-08f713e4398f-kube-api-access-mv85p\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:01:23.212573 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.212328    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/188055e8-41f9-4acf-86de-08f713e4398f-tls-certs\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:01:23.783056 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.783027    2574 generic.go:356] "Generic (PLEG): container finished" podID="188055e8-41f9-4acf-86de-08f713e4398f" containerID="2ca91e0394935ea7bebfe39edec3b6c624cddbecf122888afbc4b9e4bc2ac61b" exitCode=0
Jul 28 05:01:23.783234 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.783070    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"
Jul 28 05:01:23.783234 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.783091    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm" event={"ID":"188055e8-41f9-4acf-86de-08f713e4398f","Type":"ContainerDied","Data":"2ca91e0394935ea7bebfe39edec3b6c624cddbecf122888afbc4b9e4bc2ac61b"}
Jul 28 05:01:23.783234 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.783122    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm" event={"ID":"188055e8-41f9-4acf-86de-08f713e4398f","Type":"ContainerDied","Data":"7d323592988952821e8b3a9f1bf5a6a2f625ff2d304b834a431d7e0ee69b5bd4"}
Jul 28 05:01:23.783234 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.783138    2574 scope.go:117] "RemoveContainer" containerID="2ca91e0394935ea7bebfe39edec3b6c624cddbecf122888afbc4b9e4bc2ac61b"
Jul 28 05:01:23.791752 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.791734    2574 scope.go:117] "RemoveContainer" containerID="2ca91e0394935ea7bebfe39edec3b6c624cddbecf122888afbc4b9e4bc2ac61b"
Jul 28 05:01:23.792029 ip-10-0-133-204 kubenswrapper[2574]: E0728 05:01:23.791996    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ca91e0394935ea7bebfe39edec3b6c624cddbecf122888afbc4b9e4bc2ac61b\": container with ID starting with 2ca91e0394935ea7bebfe39edec3b6c624cddbecf122888afbc4b9e4bc2ac61b not found: ID does not exist" containerID="2ca91e0394935ea7bebfe39edec3b6c624cddbecf122888afbc4b9e4bc2ac61b"
Jul 28 05:01:23.792080 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.792039    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ca91e0394935ea7bebfe39edec3b6c624cddbecf122888afbc4b9e4bc2ac61b"} err="failed to get container status \"2ca91e0394935ea7bebfe39edec3b6c624cddbecf122888afbc4b9e4bc2ac61b\": rpc error: code = NotFound desc = could not find container \"2ca91e0394935ea7bebfe39edec3b6c624cddbecf122888afbc4b9e4bc2ac61b\": container with ID starting with 2ca91e0394935ea7bebfe39edec3b6c624cddbecf122888afbc4b9e4bc2ac61b not found: ID does not exist"
Jul 28 05:01:23.797292 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.797268    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"]
Jul 28 05:01:23.800651 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:23.800625    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/auth-test-model-92c557-kserve-64d758c9b7-c9msm"]
Jul 28 05:01:25.620218 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:25.620191    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="188055e8-41f9-4acf-86de-08f713e4398f" path="/var/lib/kubelet/pods/188055e8-41f9-4acf-86de-08f713e4398f/volumes"
Jul 28 05:01:28.314931 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:28.314880    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:01:31.482617 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:31.482574    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:01:45.858967 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:45.858937    2574 scope.go:117] "RemoveContainer" containerID="fa2b8ae14d10d034ae1f215ed0179f24d38b4005c0544ee30e2fac04c950d5a9"
Jul 28 05:01:45.867267 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:01:45.867245    2574 scope.go:117] "RemoveContainer" containerID="26e46f5eed68aef56996ec745b66c048a451ee824855d30e70250f2c2610808d"
Jul 28 05:02:04.096214 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.096172    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"]
Jul 28 05:02:04.096720 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.096556    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7063fed-a260-4312-860d-54529267118b" containerName="cleanup"
Jul 28 05:02:04.096720 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.096567    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7063fed-a260-4312-860d-54529267118b" containerName="cleanup"
Jul 28 05:02:04.096720 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.096588    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce95dfe8-f309-44b8-987e-454890e68b7b" containerName="cleanup"
Jul 28 05:02:04.096720 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.096596    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce95dfe8-f309-44b8-987e-454890e68b7b" containerName="cleanup"
Jul 28 05:02:04.096720 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.096605    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="188055e8-41f9-4acf-86de-08f713e4398f" containerName="main"
Jul 28 05:02:04.096720 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.096611    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="188055e8-41f9-4acf-86de-08f713e4398f" containerName="main"
Jul 28 05:02:04.096720 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.096617    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f427be-6c0a-4363-9332-5bca149df170" containerName="cleanup"
Jul 28 05:02:04.096720 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.096622    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f427be-6c0a-4363-9332-5bca149df170" containerName="cleanup"
Jul 28 05:02:04.096720 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.096670    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7063fed-a260-4312-860d-54529267118b" containerName="cleanup"
Jul 28 05:02:04.096720 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.096681    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="03f427be-6c0a-4363-9332-5bca149df170" containerName="cleanup"
Jul 28 05:02:04.096720 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.096688    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="188055e8-41f9-4acf-86de-08f713e4398f" containerName="main"
Jul 28 05:02:04.096720 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.096695    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce95dfe8-f309-44b8-987e-454890e68b7b" containerName="cleanup"
Jul 28 05:02:04.099774 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.099757    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.101853 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.101834    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-92c557\"/\"kube-root-ca.crt\""
Jul 28 05:02:04.102536 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.102512    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-92c557\"/\"default-dockercfg-gttgr\""
Jul 28 05:02:04.102625 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.102512    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-92c557\"/\"sub-test-model-92c557-kserve-self-signed-certs\""
Jul 28 05:02:04.102625 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.102514    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-92c557\"/\"openshift-service-ca.crt\""
Jul 28 05:02:04.110436 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.110411    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"]
Jul 28 05:02:04.189456 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.189427    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:02:04.270731 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.270687    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-dshm\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.270904 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.270737    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-tmp-dir\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.270904 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.270814    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-home\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.270904 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.270852    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-tls-certs\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.270904 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.270876    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-model-cache\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.271038 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.270911    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9d8j\" (UniqueName: \"kubernetes.io/projected/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-kube-api-access-h9d8j\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.372007 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.371909    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-tmp-dir\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.372007 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.371963    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-home\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.372007 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.371981    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-tls-certs\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.372007 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.372000    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-model-cache\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.372351 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.372042    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9d8j\" (UniqueName: \"kubernetes.io/projected/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-kube-api-access-h9d8j\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.372351 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.372107    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-dshm\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.372351 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.372336    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-tmp-dir\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.372513 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.372392    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-home\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.372513 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.372440    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-model-cache\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.374228 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.374204    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-dshm\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.374444 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.374423    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-tls-certs\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.379225 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.379202    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9d8j\" (UniqueName: \"kubernetes.io/projected/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-kube-api-access-h9d8j\") pod \"sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") " pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.409976 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.409939    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:04.540276 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.540238    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"]
Jul 28 05:02:04.542285 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:02:04.542259    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ea204b0_36e2_40ba_8ca4_1cdbf7b587f3.slice/crio-3bc88fe09d6b749372d144fcd9f74eeb8601e89916fac0389844cc181666b068 WatchSource:0}: Error finding container 3bc88fe09d6b749372d144fcd9f74eeb8601e89916fac0389844cc181666b068: Status 404 returned error can't find the container with id 3bc88fe09d6b749372d144fcd9f74eeb8601e89916fac0389844cc181666b068
Jul 28 05:02:04.983345 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.983310    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2" event={"ID":"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3","Type":"ContainerStarted","Data":"ff8aa7a83feef1e7c6c7fa13f5e9be5498277bcb598bcc6bf1e0d3f0f96787e7"}
Jul 28 05:02:04.983345 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.983352    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2" event={"ID":"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3","Type":"ContainerStarted","Data":"3bc88fe09d6b749372d144fcd9f74eeb8601e89916fac0389844cc181666b068"}
Jul 28 05:02:04.999976 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:04.999930    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2" podStartSLOduration=0.999913151 podStartE2EDuration="999.913151ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 05:02:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 05:02:04.998476471 +0000 UTC m=+3019.940316216" watchObservedRunningTime="2026-07-28 05:02:04.999913151 +0000 UTC m=+3019.941752871"
Jul 28 05:02:07.780770 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:07.780734    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:02:14.410298 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:14.410270    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:14.410850 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:14.410315    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:14.423404 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:14.423381    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:15.044235 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:15.044208    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:02:42.382676 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:02:42.382600    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:03:01.081411 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:01.081372    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:03:13.267191 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:13.267149    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:03:14.005960 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:14.005925    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"]
Jul 28 05:03:14.027755 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:14.027724    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:03:17.393449 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.393413    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"]
Jul 28 05:03:17.397006 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.396985    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.398795 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.398777    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1badbb\"/\"openshift-service-ca.crt\""
Jul 28 05:03:17.399438 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.399424    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1badbb\"/\"default-dockercfg-z2fqf\""
Jul 28 05:03:17.399438 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.399433    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1badbb\"/\"rate-test-model-1badbb-kserve-self-signed-certs\""
Jul 28 05:03:17.399571 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.399458    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1badbb\"/\"kube-root-ca.crt\""
Jul 28 05:03:17.404461 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.404439    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"]
Jul 28 05:03:17.418251 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.418227    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:03:17.470423 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.470402    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-dshm\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.470522 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.470444    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-model-cache\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.470568 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.470530    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-home\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.470626 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.470568    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-tmp-dir\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.470684 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.470619    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wckz5\" (UniqueName: \"kubernetes.io/projected/ef36ffec-310f-4922-851d-032c036056bb-kube-api-access-wckz5\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.470738 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.470680    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef36ffec-310f-4922-851d-032c036056bb-tls-certs\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.571794 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.571770    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-tmp-dir\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.571794 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.571804    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wckz5\" (UniqueName: \"kubernetes.io/projected/ef36ffec-310f-4922-851d-032c036056bb-kube-api-access-wckz5\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.571970 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.571841    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef36ffec-310f-4922-851d-032c036056bb-tls-certs\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.571970 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.571874    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-dshm\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.571970 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.571897    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-model-cache\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.571970 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.571924    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-home\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.572240 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.572221    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-tmp-dir\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.572300 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.572264    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-home\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.572300 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.572289    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-model-cache\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.574019 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.573996    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-dshm\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.574254 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.574236    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef36ffec-310f-4922-851d-032c036056bb-tls-certs\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.578444 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.578423    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wckz5\" (UniqueName: \"kubernetes.io/projected/ef36ffec-310f-4922-851d-032c036056bb-kube-api-access-wckz5\") pod \"rate-test-model-1badbb-kserve-55588d6b99-mtp9s\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") " pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.707467 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.707394    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:17.831899 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:17.831870    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"]
Jul 28 05:03:17.835725 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:03:17.835695    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef36ffec_310f_4922_851d_032c036056bb.slice/crio-c675ecc8cf18a7dc2bd5cfb228c968bb0e0c2e562231d9eaa0ca2e1f31a9422c WatchSource:0}: Error finding container c675ecc8cf18a7dc2bd5cfb228c968bb0e0c2e562231d9eaa0ca2e1f31a9422c: Status 404 returned error can't find the container with id c675ecc8cf18a7dc2bd5cfb228c968bb0e0c2e562231d9eaa0ca2e1f31a9422c
Jul 28 05:03:18.335815 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:18.335777    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s" event={"ID":"ef36ffec-310f-4922-851d-032c036056bb","Type":"ContainerStarted","Data":"82702b282a5d4f67d55100aabe16e2055a20405db18cf66efe163fba2d202b2d"}
Jul 28 05:03:18.335815 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:18.335816    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s" event={"ID":"ef36ffec-310f-4922-851d-032c036056bb","Type":"ContainerStarted","Data":"c675ecc8cf18a7dc2bd5cfb228c968bb0e0c2e562231d9eaa0ca2e1f31a9422c"}
Jul 28 05:03:18.352256 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:18.352214    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s" podStartSLOduration=1.352200339 podStartE2EDuration="1.352200339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 05:03:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 05:03:18.350480793 +0000 UTC m=+3093.292320511" watchObservedRunningTime="2026-07-28 05:03:18.352200339 +0000 UTC m=+3093.294040055"
Jul 28 05:03:25.575930 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:25.575893    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:03:27.707797 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:27.707758    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:27.708217 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:27.707814    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:27.720145 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:27.720117    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:28.396144 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:28.396117    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:03:29.024620 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.024549    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2" podUID="2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3" containerName="main" containerID="cri-o://ff8aa7a83feef1e7c6c7fa13f5e9be5498277bcb598bcc6bf1e0d3f0f96787e7" gracePeriod=45
Jul 28 05:03:29.267031 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.267000    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:03:29.367581 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.367533    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9d8j\" (UniqueName: \"kubernetes.io/projected/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-kube-api-access-h9d8j\") pod \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") "
Jul 28 05:03:29.367581 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.367580    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-model-cache\") pod \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") "
Jul 28 05:03:29.367819 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.367644    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-tmp-dir\") pod \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") "
Jul 28 05:03:29.367819 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.367689    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-home\") pod \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") "
Jul 28 05:03:29.367819 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.367710    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-tls-certs\") pod \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") "
Jul 28 05:03:29.367819 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.367736    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-dshm\") pod \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\" (UID: \"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3\") "
Jul 28 05:03:29.368032 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.367922    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-model-cache" (OuterVolumeSpecName: "model-cache") pod "2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3" (UID: "2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:03:29.368032 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.367943    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-home" (OuterVolumeSpecName: "home") pod "2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3" (UID: "2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:03:29.368141 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.368094    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3" (UID: "2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:03:29.369722 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.369695    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-kube-api-access-h9d8j" (OuterVolumeSpecName: "kube-api-access-h9d8j") pod "2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3" (UID: "2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3"). InnerVolumeSpecName "kube-api-access-h9d8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 05:03:29.369814 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.369714    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-dshm" (OuterVolumeSpecName: "dshm") pod "2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3" (UID: "2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:03:29.369814 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.369779    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3" (UID: "2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 05:03:29.392042 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.391997    2574 generic.go:356] "Generic (PLEG): container finished" podID="2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3" containerID="ff8aa7a83feef1e7c6c7fa13f5e9be5498277bcb598bcc6bf1e0d3f0f96787e7" exitCode=0
Jul 28 05:03:29.392042 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.392041    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"
Jul 28 05:03:29.392213 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.392085    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2" event={"ID":"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3","Type":"ContainerDied","Data":"ff8aa7a83feef1e7c6c7fa13f5e9be5498277bcb598bcc6bf1e0d3f0f96787e7"}
Jul 28 05:03:29.392213 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.392128    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2" event={"ID":"2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3","Type":"ContainerDied","Data":"3bc88fe09d6b749372d144fcd9f74eeb8601e89916fac0389844cc181666b068"}
Jul 28 05:03:29.392213 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.392148    2574 scope.go:117] "RemoveContainer" containerID="ff8aa7a83feef1e7c6c7fa13f5e9be5498277bcb598bcc6bf1e0d3f0f96787e7"
Jul 28 05:03:29.400939 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.400921    2574 scope.go:117] "RemoveContainer" containerID="ff8aa7a83feef1e7c6c7fa13f5e9be5498277bcb598bcc6bf1e0d3f0f96787e7"
Jul 28 05:03:29.401184 ip-10-0-133-204 kubenswrapper[2574]: E0728 05:03:29.401165    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff8aa7a83feef1e7c6c7fa13f5e9be5498277bcb598bcc6bf1e0d3f0f96787e7\": container with ID starting with ff8aa7a83feef1e7c6c7fa13f5e9be5498277bcb598bcc6bf1e0d3f0f96787e7 not found: ID does not exist" containerID="ff8aa7a83feef1e7c6c7fa13f5e9be5498277bcb598bcc6bf1e0d3f0f96787e7"
Jul 28 05:03:29.401244 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.401197    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8aa7a83feef1e7c6c7fa13f5e9be5498277bcb598bcc6bf1e0d3f0f96787e7"} err="failed to get container status \"ff8aa7a83feef1e7c6c7fa13f5e9be5498277bcb598bcc6bf1e0d3f0f96787e7\": rpc error: code = NotFound desc = could not find container \"ff8aa7a83feef1e7c6c7fa13f5e9be5498277bcb598bcc6bf1e0d3f0f96787e7\": container with ID starting with ff8aa7a83feef1e7c6c7fa13f5e9be5498277bcb598bcc6bf1e0d3f0f96787e7 not found: ID does not exist"
Jul 28 05:03:29.411050 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.411020    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"]
Jul 28 05:03:29.414194 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.414172    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/sub-test-model-92c557-kserve-6cd4f99dfc-4mhz2"]
Jul 28 05:03:29.468338 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.468307    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-home\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:03:29.468338 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.468331    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-tls-certs\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:03:29.468338 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.468340    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-dshm\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:03:29.468617 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.468348    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9d8j\" (UniqueName: \"kubernetes.io/projected/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-kube-api-access-h9d8j\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:03:29.468617 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.468355    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-model-cache\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:03:29.468617 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.468380    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3-tmp-dir\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:03:29.620520 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:29.620444    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3" path="/var/lib/kubelet/pods/2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3/volumes"
Jul 28 05:03:41.689020 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.688989    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"]
Jul 28 05:03:41.689487 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.689415    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3" containerName="main"
Jul 28 05:03:41.689487 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.689427    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3" containerName="main"
Jul 28 05:03:41.689603 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.689492    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ea204b0-36e2-40ba-8ca4-1cdbf7b587f3" containerName="main"
Jul 28 05:03:41.693832 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.693814    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.695714 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.695696    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-92c557\"/\"openshift-service-ca.crt\""
Jul 28 05:03:41.696386 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.696368    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-92c557\"/\"rate-test-model-92c557-kserve-self-signed-certs\""
Jul 28 05:03:41.696458 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.696385    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-92c557\"/\"default-dockercfg-gttgr\""
Jul 28 05:03:41.696458 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.696389    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-92c557\"/\"kube-root-ca.crt\""
Jul 28 05:03:41.702639 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.702617    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"]
Jul 28 05:03:41.715920 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.715887    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:03:41.773712 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.773681    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-home\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.773712 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.773715    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-dshm\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.773905 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.773750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gphnz\" (UniqueName: \"kubernetes.io/projected/c3bec1e3-8998-4053-8015-c45184a9920e-kube-api-access-gphnz\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.773905 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.773820    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-tmp-dir\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.773905 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.773891    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-model-cache\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.774016 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.773916    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3bec1e3-8998-4053-8015-c45184a9920e-tls-certs\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.874637 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.874606    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-tmp-dir\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.874780 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.874657    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-model-cache\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.874780 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.874687    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3bec1e3-8998-4053-8015-c45184a9920e-tls-certs\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.874780 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.874734    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-home\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.874780 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.874756    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-dshm\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.874915 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.874805    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gphnz\" (UniqueName: \"kubernetes.io/projected/c3bec1e3-8998-4053-8015-c45184a9920e-kube-api-access-gphnz\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.875095 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.875060    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-tmp-dir\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.875095 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.875080    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-model-cache\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.875189 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.875154    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-home\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.876929 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.876904    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-dshm\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.877219 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.877186    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3bec1e3-8998-4053-8015-c45184a9920e-tls-certs\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:41.882767 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:41.882744    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gphnz\" (UniqueName: \"kubernetes.io/projected/c3bec1e3-8998-4053-8015-c45184a9920e-kube-api-access-gphnz\") pod \"rate-test-model-92c557-kserve-8699bc9f95-l8jqp\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") " pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:42.003164 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:42.003077    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:42.130297 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:42.130272    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"]
Jul 28 05:03:42.132323 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:03:42.132297    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3bec1e3_8998_4053_8015_c45184a9920e.slice/crio-92d792b33900eb104fdd9449d8cd9445595b83381e86dc1883784e611c107104 WatchSource:0}: Error finding container 92d792b33900eb104fdd9449d8cd9445595b83381e86dc1883784e611c107104: Status 404 returned error can't find the container with id 92d792b33900eb104fdd9449d8cd9445595b83381e86dc1883784e611c107104
Jul 28 05:03:42.457194 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:42.457159    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp" event={"ID":"c3bec1e3-8998-4053-8015-c45184a9920e","Type":"ContainerStarted","Data":"5c649682d20e0ff96460c70e1ea4452f54d768da2027fb0728b2320d84959b55"}
Jul 28 05:03:42.457194 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:42.457197    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp" event={"ID":"c3bec1e3-8998-4053-8015-c45184a9920e","Type":"ContainerStarted","Data":"92d792b33900eb104fdd9449d8cd9445595b83381e86dc1883784e611c107104"}
Jul 28 05:03:42.473369 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:42.473310    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp" podStartSLOduration=1.4732958219999999 podStartE2EDuration="1.473295822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 05:03:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 05:03:42.471448493 +0000 UTC m=+3117.413288211" watchObservedRunningTime="2026-07-28 05:03:42.473295822 +0000 UTC m=+3117.415135540"
Jul 28 05:03:45.475514 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:45.475480    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:03:47.291570 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:47.291537    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:03:48.716431 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:48.716387    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:03:52.004059 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:52.004010    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:52.004059 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:52.004068    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:52.016514 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:52.016487    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:03:52.515304 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:03:52.515275    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:04:22.686256 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:04:22.686215    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:04:36.982408 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:04:36.982354    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:04:51.888104 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:04:51.888066    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:04:55.768434 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:04:55.768397    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"]
Jul 28 05:04:55.787281 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:04:55.787252    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:04:56.602784 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:04:56.602751    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"]
Jul 28 05:04:56.620523 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:04:56.620490    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:08.279833 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:08.279795    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:05:10.788111 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:10.788047    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s" podUID="ef36ffec-310f-4922-851d-032c036056bb" containerName="main" containerID="cri-o://82702b282a5d4f67d55100aabe16e2055a20405db18cf66efe163fba2d202b2d" gracePeriod=45
Jul 28 05:05:10.912968 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:10.912928    2574 generic.go:356] "Generic (PLEG): container finished" podID="ef36ffec-310f-4922-851d-032c036056bb" containerID="82702b282a5d4f67d55100aabe16e2055a20405db18cf66efe163fba2d202b2d" exitCode=0
Jul 28 05:05:10.913097 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:10.912992    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s" event={"ID":"ef36ffec-310f-4922-851d-032c036056bb","Type":"ContainerDied","Data":"82702b282a5d4f67d55100aabe16e2055a20405db18cf66efe163fba2d202b2d"}
Jul 28 05:05:11.039397 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.039327    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:05:11.134054 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.134020    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef36ffec-310f-4922-851d-032c036056bb-tls-certs\") pod \"ef36ffec-310f-4922-851d-032c036056bb\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") "
Jul 28 05:05:11.134054 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.134058    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-dshm\") pod \"ef36ffec-310f-4922-851d-032c036056bb\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") "
Jul 28 05:05:11.134380 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.134098    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wckz5\" (UniqueName: \"kubernetes.io/projected/ef36ffec-310f-4922-851d-032c036056bb-kube-api-access-wckz5\") pod \"ef36ffec-310f-4922-851d-032c036056bb\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") "
Jul 28 05:05:11.134380 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.134151    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-home\") pod \"ef36ffec-310f-4922-851d-032c036056bb\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") "
Jul 28 05:05:11.134380 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.134187    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-tmp-dir\") pod \"ef36ffec-310f-4922-851d-032c036056bb\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") "
Jul 28 05:05:11.134380 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.134221    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-model-cache\") pod \"ef36ffec-310f-4922-851d-032c036056bb\" (UID: \"ef36ffec-310f-4922-851d-032c036056bb\") "
Jul 28 05:05:11.134609 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.134428    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-home" (OuterVolumeSpecName: "home") pod "ef36ffec-310f-4922-851d-032c036056bb" (UID: "ef36ffec-310f-4922-851d-032c036056bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:05:11.134609 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.134494    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ef36ffec-310f-4922-851d-032c036056bb" (UID: "ef36ffec-310f-4922-851d-032c036056bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:05:11.134699 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.134653    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-model-cache" (OuterVolumeSpecName: "model-cache") pod "ef36ffec-310f-4922-851d-032c036056bb" (UID: "ef36ffec-310f-4922-851d-032c036056bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:05:11.136342 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.136320    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef36ffec-310f-4922-851d-032c036056bb-kube-api-access-wckz5" (OuterVolumeSpecName: "kube-api-access-wckz5") pod "ef36ffec-310f-4922-851d-032c036056bb" (UID: "ef36ffec-310f-4922-851d-032c036056bb"). InnerVolumeSpecName "kube-api-access-wckz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 05:05:11.136609 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.136583    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef36ffec-310f-4922-851d-032c036056bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ef36ffec-310f-4922-851d-032c036056bb" (UID: "ef36ffec-310f-4922-851d-032c036056bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 05:05:11.136609 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.136603    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-dshm" (OuterVolumeSpecName: "dshm") pod "ef36ffec-310f-4922-851d-032c036056bb" (UID: "ef36ffec-310f-4922-851d-032c036056bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:05:11.235759 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.235726    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-home\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:05:11.235759 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.235755    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-tmp-dir\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:05:11.235759 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.235766    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-model-cache\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:05:11.235980 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.235774    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef36ffec-310f-4922-851d-032c036056bb-tls-certs\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:05:11.235980 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.235782    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef36ffec-310f-4922-851d-032c036056bb-dshm\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:05:11.235980 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.235790    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wckz5\" (UniqueName: \"kubernetes.io/projected/ef36ffec-310f-4922-851d-032c036056bb-kube-api-access-wckz5\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:05:11.623631 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.623589    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp" podUID="c3bec1e3-8998-4053-8015-c45184a9920e" containerName="main" containerID="cri-o://5c649682d20e0ff96460c70e1ea4452f54d768da2027fb0728b2320d84959b55" gracePeriod=45
Jul 28 05:05:11.874100 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.874046    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:05:11.919369 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.919327    2574 generic.go:356] "Generic (PLEG): container finished" podID="c3bec1e3-8998-4053-8015-c45184a9920e" containerID="5c649682d20e0ff96460c70e1ea4452f54d768da2027fb0728b2320d84959b55" exitCode=0
Jul 28 05:05:11.919521 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.919383    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"
Jul 28 05:05:11.919521 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.919413    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp" event={"ID":"c3bec1e3-8998-4053-8015-c45184a9920e","Type":"ContainerDied","Data":"5c649682d20e0ff96460c70e1ea4452f54d768da2027fb0728b2320d84959b55"}
Jul 28 05:05:11.919521 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.919456    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp" event={"ID":"c3bec1e3-8998-4053-8015-c45184a9920e","Type":"ContainerDied","Data":"92d792b33900eb104fdd9449d8cd9445595b83381e86dc1883784e611c107104"}
Jul 28 05:05:11.919521 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.919476    2574 scope.go:117] "RemoveContainer" containerID="5c649682d20e0ff96460c70e1ea4452f54d768da2027fb0728b2320d84959b55"
Jul 28 05:05:11.920813 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.920792    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s" event={"ID":"ef36ffec-310f-4922-851d-032c036056bb","Type":"ContainerDied","Data":"c675ecc8cf18a7dc2bd5cfb228c968bb0e0c2e562231d9eaa0ca2e1f31a9422c"}
Jul 28 05:05:11.920927 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.920827    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"
Jul 28 05:05:11.928771 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.928754    2574 scope.go:117] "RemoveContainer" containerID="5c649682d20e0ff96460c70e1ea4452f54d768da2027fb0728b2320d84959b55"
Jul 28 05:05:11.929041 ip-10-0-133-204 kubenswrapper[2574]: E0728 05:05:11.929021    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c649682d20e0ff96460c70e1ea4452f54d768da2027fb0728b2320d84959b55\": container with ID starting with 5c649682d20e0ff96460c70e1ea4452f54d768da2027fb0728b2320d84959b55 not found: ID does not exist" containerID="5c649682d20e0ff96460c70e1ea4452f54d768da2027fb0728b2320d84959b55"
Jul 28 05:05:11.929133 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.929052    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c649682d20e0ff96460c70e1ea4452f54d768da2027fb0728b2320d84959b55"} err="failed to get container status \"5c649682d20e0ff96460c70e1ea4452f54d768da2027fb0728b2320d84959b55\": rpc error: code = NotFound desc = could not find container \"5c649682d20e0ff96460c70e1ea4452f54d768da2027fb0728b2320d84959b55\": container with ID starting with 5c649682d20e0ff96460c70e1ea4452f54d768da2027fb0728b2320d84959b55 not found: ID does not exist"
Jul 28 05:05:11.929133 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.929076    2574 scope.go:117] "RemoveContainer" containerID="82702b282a5d4f67d55100aabe16e2055a20405db18cf66efe163fba2d202b2d"
Jul 28 05:05:11.938320 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.938293    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"]
Jul 28 05:05:11.939983 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.939960    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1badbb/rate-test-model-1badbb-kserve-55588d6b99-mtp9s"]
Jul 28 05:05:11.940642 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.940625    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3bec1e3-8998-4053-8015-c45184a9920e-tls-certs\") pod \"c3bec1e3-8998-4053-8015-c45184a9920e\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") "
Jul 28 05:05:11.940739 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.940724    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-tmp-dir\") pod \"c3bec1e3-8998-4053-8015-c45184a9920e\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") "
Jul 28 05:05:11.940778 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.940764    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-dshm\") pod \"c3bec1e3-8998-4053-8015-c45184a9920e\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") "
Jul 28 05:05:11.940828 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.940815    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gphnz\" (UniqueName: \"kubernetes.io/projected/c3bec1e3-8998-4053-8015-c45184a9920e-kube-api-access-gphnz\") pod \"c3bec1e3-8998-4053-8015-c45184a9920e\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") "
Jul 28 05:05:11.940877 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.940841    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-model-cache\") pod \"c3bec1e3-8998-4053-8015-c45184a9920e\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") "
Jul 28 05:05:11.940928 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.940884    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-home\") pod \"c3bec1e3-8998-4053-8015-c45184a9920e\" (UID: \"c3bec1e3-8998-4053-8015-c45184a9920e\") "
Jul 28 05:05:11.941021 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.940982    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c3bec1e3-8998-4053-8015-c45184a9920e" (UID: "c3bec1e3-8998-4053-8015-c45184a9920e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:05:11.941147 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.941107    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-model-cache" (OuterVolumeSpecName: "model-cache") pod "c3bec1e3-8998-4053-8015-c45184a9920e" (UID: "c3bec1e3-8998-4053-8015-c45184a9920e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:05:11.941223 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.941154    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-tmp-dir\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:05:11.941223 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.941162    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-home" (OuterVolumeSpecName: "home") pod "c3bec1e3-8998-4053-8015-c45184a9920e" (UID: "c3bec1e3-8998-4053-8015-c45184a9920e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:05:11.942685 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.942664    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3bec1e3-8998-4053-8015-c45184a9920e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c3bec1e3-8998-4053-8015-c45184a9920e" (UID: "c3bec1e3-8998-4053-8015-c45184a9920e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 05:05:11.942769 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.942690    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-dshm" (OuterVolumeSpecName: "dshm") pod "c3bec1e3-8998-4053-8015-c45184a9920e" (UID: "c3bec1e3-8998-4053-8015-c45184a9920e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:05:11.942824 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:11.942779    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3bec1e3-8998-4053-8015-c45184a9920e-kube-api-access-gphnz" (OuterVolumeSpecName: "kube-api-access-gphnz") pod "c3bec1e3-8998-4053-8015-c45184a9920e" (UID: "c3bec1e3-8998-4053-8015-c45184a9920e"). InnerVolumeSpecName "kube-api-access-gphnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 05:05:12.042067 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:12.042037    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-home\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:05:12.042067 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:12.042062    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3bec1e3-8998-4053-8015-c45184a9920e-tls-certs\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:05:12.042067 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:12.042072    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-dshm\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:05:12.042272 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:12.042082    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gphnz\" (UniqueName: \"kubernetes.io/projected/c3bec1e3-8998-4053-8015-c45184a9920e-kube-api-access-gphnz\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:05:12.042272 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:12.042092    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3bec1e3-8998-4053-8015-c45184a9920e-model-cache\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:05:12.239197 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:12.239167    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"]
Jul 28 05:05:12.243350 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:12.243323    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-92c557/rate-test-model-92c557-kserve-8699bc9f95-l8jqp"]
Jul 28 05:05:13.619806 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:13.619773    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3bec1e3-8998-4053-8015-c45184a9920e" path="/var/lib/kubelet/pods/c3bec1e3-8998-4053-8015-c45184a9920e/volumes"
Jul 28 05:05:13.620268 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:13.620165    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef36ffec-310f-4922-851d-032c036056bb" path="/var/lib/kubelet/pods/ef36ffec-310f-4922-851d-032c036056bb/volumes"
Jul 28 05:05:15.112664 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:15.112632    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:15.430971 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:15.430887    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:15.825847 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:15.825817    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:19.577444 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:19.577409    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:19.634783 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:19.634754    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:22.986531 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:22.986493    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:27.685964 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:27.685925    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:05:33.971843 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:33.971808    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:35.470622 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:35.470553    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:37.152603 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:37.152565    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:37.252574 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:37.252538    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:37.408658 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:37.408547    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:37.779028 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:37.779001    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:38.041940 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:38.041819    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:38.138814 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:38.138772    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:41.721395 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:41.721327    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:05:46.681376 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:05:46.681334    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:06:15.471315 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.471277    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"]
Jul 28 05:06:15.471928 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.471762    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3bec1e3-8998-4053-8015-c45184a9920e" containerName="main"
Jul 28 05:06:15.471928 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.471781    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3bec1e3-8998-4053-8015-c45184a9920e" containerName="main"
Jul 28 05:06:15.471928 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.471811    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef36ffec-310f-4922-851d-032c036056bb" containerName="main"
Jul 28 05:06:15.471928 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.471819    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef36ffec-310f-4922-851d-032c036056bb" containerName="main"
Jul 28 05:06:15.471928 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.471905    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef36ffec-310f-4922-851d-032c036056bb" containerName="main"
Jul 28 05:06:15.471928 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.471919    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3bec1e3-8998-4053-8015-c45184a9920e" containerName="main"
Jul 28 05:06:15.475143 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.475122    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.477104 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.477084    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-dc1efb\"/\"openshift-service-ca.crt\""
Jul 28 05:06:15.477734 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.477687    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-dc1efb\"/\"kube-root-ca.crt\""
Jul 28 05:06:15.477847 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.477697    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-dc1efb\"/\"ipp-route-dc1efb-kserve-self-signed-certs\""
Jul 28 05:06:15.477847 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.477794    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-dc1efb\"/\"default-dockercfg-68qmf\""
Jul 28 05:06:15.483672 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.483647    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"]
Jul 28 05:06:15.504576 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.504525    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:06:15.646751 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.646720    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/641a25c9-f006-49fb-8b9b-527d0975dc82-tls-certs\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.646751 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.646753    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-home\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.646957 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.646779    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-dshm\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.646957 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.646810    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2t9r\" (UniqueName: \"kubernetes.io/projected/641a25c9-f006-49fb-8b9b-527d0975dc82-kube-api-access-z2t9r\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.646957 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.646869    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-model-cache\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.646957 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.646888    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-tmp-dir\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.747792 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.747716    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/641a25c9-f006-49fb-8b9b-527d0975dc82-tls-certs\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.747792 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.747749    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-home\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.747792 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.747791    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-dshm\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.748036 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.747905    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2t9r\" (UniqueName: \"kubernetes.io/projected/641a25c9-f006-49fb-8b9b-527d0975dc82-kube-api-access-z2t9r\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.748036 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.747987    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-model-cache\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.748036 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.748017    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-tmp-dir\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.748220 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.748191    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-home\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.748349 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.748333    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-model-cache\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.748424 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.748406    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-tmp-dir\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.750009 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.749981    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-dshm\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.750403 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.750384    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/641a25c9-f006-49fb-8b9b-527d0975dc82-tls-certs\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.758100 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.758076    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2t9r\" (UniqueName: \"kubernetes.io/projected/641a25c9-f006-49fb-8b9b-527d0975dc82-kube-api-access-z2t9r\") pod \"ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") " pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.786474 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.786455    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:15.916337 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:15.916313    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"]
Jul 28 05:06:15.918449 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:06:15.918418    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod641a25c9_f006_49fb_8b9b_527d0975dc82.slice/crio-4ba7a5de849a4cfc9e18c6fcff5d1eb341f8f9262e03c0f474d97682290b19b1 WatchSource:0}: Error finding container 4ba7a5de849a4cfc9e18c6fcff5d1eb341f8f9262e03c0f474d97682290b19b1: Status 404 returned error can't find the container with id 4ba7a5de849a4cfc9e18c6fcff5d1eb341f8f9262e03c0f474d97682290b19b1
Jul 28 05:06:16.233427 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:16.233390    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l" event={"ID":"641a25c9-f006-49fb-8b9b-527d0975dc82","Type":"ContainerStarted","Data":"9fd07612511ab4685142b23bc291cda062dfa81099b74994ab6d9239369297f8"}
Jul 28 05:06:16.233632 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:16.233439    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l" event={"ID":"641a25c9-f006-49fb-8b9b-527d0975dc82","Type":"ContainerStarted","Data":"4ba7a5de849a4cfc9e18c6fcff5d1eb341f8f9262e03c0f474d97682290b19b1"}
Jul 28 05:06:16.247933 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:16.247874    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l" podStartSLOduration=1.247852554 podStartE2EDuration="1.247852554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 05:06:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 05:06:16.247539325 +0000 UTC m=+3271.189379048" watchObservedRunningTime="2026-07-28 05:06:16.247852554 +0000 UTC m=+3271.189692275"
Jul 28 05:06:19.379706 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:19.379673    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:06:25.786915 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:25.786879    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:25.786915 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:25.786918    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:25.799021 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:25.798997    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:26.294193 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:26.294166    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:06:28.835721 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:28.835691    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:06:44.379513 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:44.379477    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:06:57.210273 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:57.210240    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:06:57.514847 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:57.514764    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:06:57.710668 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:06:57.710632    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:00.967242 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:00.967207    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:01.067113 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:01.067085    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:01.254906 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:01.254828    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:01.357592 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:01.357563    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:01.855293 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:01.855262    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:01.957868 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:01.957838    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:05.494737 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:05.494704    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:14.285337 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:14.285297    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:07:16.492462 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:16.492424    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2"]
Jul 28 05:07:16.496029 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:16.496010    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2"
Jul 28 05:07:16.498084 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:16.498048    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 05:07:16.498084 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:16.498072    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 05:07:16.501219 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:16.501194    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2"]
Jul 28 05:07:16.637848 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:16.637812    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff582764-8b79-4f02-8c18-495837cc6d96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2\" (UID: \"ff582764-8b79-4f02-8c18-495837cc6d96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2"
Jul 28 05:07:16.738786 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:16.738750    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff582764-8b79-4f02-8c18-495837cc6d96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2\" (UID: \"ff582764-8b79-4f02-8c18-495837cc6d96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2"
Jul 28 05:07:16.739594 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:16.739566    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff582764-8b79-4f02-8c18-495837cc6d96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2\" (UID: \"ff582764-8b79-4f02-8c18-495837cc6d96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2"
Jul 28 05:07:16.808548 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:16.808510    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2"
Jul 28 05:07:16.944353 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:16.944327    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2"]
Jul 28 05:07:16.946782 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:07:16.946751    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff582764_8b79_4f02_8c18_495837cc6d96.slice/crio-25b454e0bf473c3f4d23a2c0850838314aa64ae956ea7bd2cdf757ef7f2c4cc1 WatchSource:0}: Error finding container 25b454e0bf473c3f4d23a2c0850838314aa64ae956ea7bd2cdf757ef7f2c4cc1: Status 404 returned error can't find the container with id 25b454e0bf473c3f4d23a2c0850838314aa64ae956ea7bd2cdf757ef7f2c4cc1
Jul 28 05:07:17.547109 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:17.547076    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2" event={"ID":"ff582764-8b79-4f02-8c18-495837cc6d96","Type":"ContainerStarted","Data":"8367305f4d7372118618e881527658499c10d7b6b689ee77d6eb80910056f67c"}
Jul 28 05:07:17.547485 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:17.547140    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2" event={"ID":"ff582764-8b79-4f02-8c18-495837cc6d96","Type":"ContainerStarted","Data":"25b454e0bf473c3f4d23a2c0850838314aa64ae956ea7bd2cdf757ef7f2c4cc1"}
Jul 28 05:07:17.561847 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:17.561798    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2" podStartSLOduration=1.561785037 podStartE2EDuration="1.561785037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 05:07:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 05:07:17.559243037 +0000 UTC m=+3332.501082940" watchObservedRunningTime="2026-07-28 05:07:17.561785037 +0000 UTC m=+3332.503624757"
Jul 28 05:07:18.553841 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:18.553805    2574 generic.go:356] "Generic (PLEG): container finished" podID="ff582764-8b79-4f02-8c18-495837cc6d96" containerID="8367305f4d7372118618e881527658499c10d7b6b689ee77d6eb80910056f67c" exitCode=0
Jul 28 05:07:18.554258 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:18.553894    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2" event={"ID":"ff582764-8b79-4f02-8c18-495837cc6d96","Type":"ContainerDied","Data":"8367305f4d7372118618e881527658499c10d7b6b689ee77d6eb80910056f67c"}
Jul 28 05:07:19.687765 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:19.687743    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2"
Jul 28 05:07:19.761650 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:19.761624    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff582764-8b79-4f02-8c18-495837cc6d96-maas-api-service-ca\") pod \"ff582764-8b79-4f02-8c18-495837cc6d96\" (UID: \"ff582764-8b79-4f02-8c18-495837cc6d96\") "
Jul 28 05:07:19.761929 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:19.761909    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff582764-8b79-4f02-8c18-495837cc6d96-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff582764-8b79-4f02-8c18-495837cc6d96" (UID: "ff582764-8b79-4f02-8c18-495837cc6d96"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 05:07:19.862421 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:19.862352    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff582764-8b79-4f02-8c18-495837cc6d96-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:07:20.564783 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:20.564754    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2"
Jul 28 05:07:20.564783 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:20.564769    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2" event={"ID":"ff582764-8b79-4f02-8c18-495837cc6d96","Type":"ContainerDied","Data":"25b454e0bf473c3f4d23a2c0850838314aa64ae956ea7bd2cdf757ef7f2c4cc1"}
Jul 28 05:07:20.565003 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:20.564810    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25b454e0bf473c3f4d23a2c0850838314aa64ae956ea7bd2cdf757ef7f2c4cc1"
Jul 28 05:07:27.061537 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:27.061480    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:27.131206 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:27.131176    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:27.194234 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:27.194205    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:31.571244 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:31.571205    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2"]
Jul 28 05:07:31.572982 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:31.572955    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:31.573260 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:31.573238    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-29b6e5-6kkq2"]
Jul 28 05:07:31.619697 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:31.619672    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff582764-8b79-4f02-8c18-495837cc6d96" path="/var/lib/kubelet/pods/ff582764-8b79-4f02-8c18-495837cc6d96/volumes"
Jul 28 05:07:31.963500 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:31.963432    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:34.574713 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:34.574678    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:07:37.688963 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:37.688902    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:37.745439 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:37.745404    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:37.967693 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:37.967607    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:38.215730 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:38.215694    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:41.842316 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:41.842284    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k"]
Jul 28 05:07:41.842689 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:41.842655    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff582764-8b79-4f02-8c18-495837cc6d96" containerName="revoke-keys"
Jul 28 05:07:41.842689 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:41.842667    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff582764-8b79-4f02-8c18-495837cc6d96" containerName="revoke-keys"
Jul 28 05:07:41.842764 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:41.842736    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff582764-8b79-4f02-8c18-495837cc6d96" containerName="revoke-keys"
Jul 28 05:07:41.845793 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:41.845777    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k"
Jul 28 05:07:41.847803 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:41.847781    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 05:07:41.847933 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:41.847871    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 05:07:41.851006 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:41.850985    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k"]
Jul 28 05:07:41.938269 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:41.938246    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc508d1-f541-41d7-bf6f-69038dcf56b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k\" (UID: \"dbc508d1-f541-41d7-bf6f-69038dcf56b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k"
Jul 28 05:07:42.038944 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.038920    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc508d1-f541-41d7-bf6f-69038dcf56b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k\" (UID: \"dbc508d1-f541-41d7-bf6f-69038dcf56b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k"
Jul 28 05:07:42.039507 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.039489    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc508d1-f541-41d7-bf6f-69038dcf56b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k\" (UID: \"dbc508d1-f541-41d7-bf6f-69038dcf56b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k"
Jul 28 05:07:42.157719 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.157672    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k"
Jul 28 05:07:42.282790 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.282763    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k"]
Jul 28 05:07:42.285198 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:07:42.285157    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc508d1_f541_41d7_bf6f_69038dcf56b3.slice/crio-3d83ad47a5f2b8a77aa0aa616e37eac87182cefec34c344d1049c0e4a53b14af WatchSource:0}: Error finding container 3d83ad47a5f2b8a77aa0aa616e37eac87182cefec34c344d1049c0e4a53b14af: Status 404 returned error can't find the container with id 3d83ad47a5f2b8a77aa0aa616e37eac87182cefec34c344d1049c0e4a53b14af
Jul 28 05:07:42.486167 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.486144    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vwcdr"]
Jul 28 05:07:42.489613 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.489591    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr"
Jul 28 05:07:42.491600 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.491582    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-jl2fp\""
Jul 28 05:07:42.499382 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.499336    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vwcdr"]
Jul 28 05:07:42.543710 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.543687    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49z2v\" (UniqueName: \"kubernetes.io/projected/b13e2930-e5d9-40d6-844a-4525d3733f98-kube-api-access-49z2v\") pod \"kuadrant-operator-catalog-vwcdr\" (UID: \"b13e2930-e5d9-40d6-844a-4525d3733f98\") " pod="kuadrant-system/kuadrant-operator-catalog-vwcdr"
Jul 28 05:07:42.644699 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.644678    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49z2v\" (UniqueName: \"kubernetes.io/projected/b13e2930-e5d9-40d6-844a-4525d3733f98-kube-api-access-49z2v\") pod \"kuadrant-operator-catalog-vwcdr\" (UID: \"b13e2930-e5d9-40d6-844a-4525d3733f98\") " pod="kuadrant-system/kuadrant-operator-catalog-vwcdr"
Jul 28 05:07:42.651745 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.651724    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49z2v\" (UniqueName: \"kubernetes.io/projected/b13e2930-e5d9-40d6-844a-4525d3733f98-kube-api-access-49z2v\") pod \"kuadrant-operator-catalog-vwcdr\" (UID: \"b13e2930-e5d9-40d6-844a-4525d3733f98\") " pod="kuadrant-system/kuadrant-operator-catalog-vwcdr"
Jul 28 05:07:42.672023 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.671991    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k" event={"ID":"dbc508d1-f541-41d7-bf6f-69038dcf56b3","Type":"ContainerStarted","Data":"9ae04b09d379175db4b6422df589eed67dcf74ab6a9fa147197603c8af82a2bb"}
Jul 28 05:07:42.672119 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.672048    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k" event={"ID":"dbc508d1-f541-41d7-bf6f-69038dcf56b3","Type":"ContainerStarted","Data":"3d83ad47a5f2b8a77aa0aa616e37eac87182cefec34c344d1049c0e4a53b14af"}
Jul 28 05:07:42.687109 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.687069    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k" podStartSLOduration=1.687055386 podStartE2EDuration="1.687055386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 05:07:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 05:07:42.68409742 +0000 UTC m=+3357.625937141" watchObservedRunningTime="2026-07-28 05:07:42.687055386 +0000 UTC m=+3357.628895106"
Jul 28 05:07:42.799079 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.799058    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr"
Jul 28 05:07:42.924132 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.924107    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vwcdr"]
Jul 28 05:07:42.925726 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:07:42.925703    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb13e2930_e5d9_40d6_844a_4525d3733f98.slice/crio-ad8a81686eb9147204db53e338c5398c1e5f21ba0332fe5b0cbdb4f0eba660e0 WatchSource:0}: Error finding container ad8a81686eb9147204db53e338c5398c1e5f21ba0332fe5b0cbdb4f0eba660e0: Status 404 returned error can't find the container with id ad8a81686eb9147204db53e338c5398c1e5f21ba0332fe5b0cbdb4f0eba660e0
Jul 28 05:07:42.927115 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:42.927098    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 05:07:43.681740 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:43.681705    2574 generic.go:356] "Generic (PLEG): container finished" podID="dbc508d1-f541-41d7-bf6f-69038dcf56b3" containerID="9ae04b09d379175db4b6422df589eed67dcf74ab6a9fa147197603c8af82a2bb" exitCode=0
Jul 28 05:07:43.681929 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:43.681844    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k" event={"ID":"dbc508d1-f541-41d7-bf6f-69038dcf56b3","Type":"ContainerDied","Data":"9ae04b09d379175db4b6422df589eed67dcf74ab6a9fa147197603c8af82a2bb"}
Jul 28 05:07:43.683347 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:43.683305    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr" event={"ID":"b13e2930-e5d9-40d6-844a-4525d3733f98","Type":"ContainerStarted","Data":"ad8a81686eb9147204db53e338c5398c1e5f21ba0332fe5b0cbdb4f0eba660e0"}
Jul 28 05:07:44.689539 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:44.689458    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr" event={"ID":"b13e2930-e5d9-40d6-844a-4525d3733f98","Type":"ContainerStarted","Data":"be475e92391cb03988fa411aec321cd40625ccf532adb3a98082842749961a3d"}
Jul 28 05:07:44.703052 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:44.703001    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr" podStartSLOduration=1.217520998 podStartE2EDuration="2.702982231s" totalImagesPullingTime="1.485461233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 05:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 05:07:44.702318743 +0000 UTC m=+3359.644158464" watchObservedRunningTime="2026-07-28 05:07:44.702982231 +0000 UTC m=+3359.644821951"
Jul 28 05:07:44.815298 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:44.815274    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k"
Jul 28 05:07:44.862977 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:44.862952    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc508d1-f541-41d7-bf6f-69038dcf56b3-maas-api-service-ca\") pod \"dbc508d1-f541-41d7-bf6f-69038dcf56b3\" (UID: \"dbc508d1-f541-41d7-bf6f-69038dcf56b3\") "
Jul 28 05:07:44.863271 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:44.863248    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbc508d1-f541-41d7-bf6f-69038dcf56b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dbc508d1-f541-41d7-bf6f-69038dcf56b3" (UID: "dbc508d1-f541-41d7-bf6f-69038dcf56b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 05:07:44.963546 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:44.963497    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc508d1-f541-41d7-bf6f-69038dcf56b3-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:07:45.695351 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:45.695326    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k"
Jul 28 05:07:45.695669 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:45.695373    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k" event={"ID":"dbc508d1-f541-41d7-bf6f-69038dcf56b3","Type":"ContainerDied","Data":"3d83ad47a5f2b8a77aa0aa616e37eac87182cefec34c344d1049c0e4a53b14af"}
Jul 28 05:07:45.695669 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:45.695419    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d83ad47a5f2b8a77aa0aa616e37eac87182cefec34c344d1049c0e4a53b14af"
Jul 28 05:07:52.799348 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:52.799318    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr"
Jul 28 05:07:52.799764 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:52.799426    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr"
Jul 28 05:07:52.819822 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:52.819795    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr"
Jul 28 05:07:53.754390 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:53.754343    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr"
Jul 28 05:07:53.785195 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:53.785164    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vwcdr"]
Jul 28 05:07:55.742908 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:55.742872    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr" podUID="b13e2930-e5d9-40d6-844a-4525d3733f98" containerName="registry-server" containerID="cri-o://be475e92391cb03988fa411aec321cd40625ccf532adb3a98082842749961a3d" gracePeriod=2
Jul 28 05:07:55.991311 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:55.991288    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr"
Jul 28 05:07:56.051685 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:56.051663    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49z2v\" (UniqueName: \"kubernetes.io/projected/b13e2930-e5d9-40d6-844a-4525d3733f98-kube-api-access-49z2v\") pod \"b13e2930-e5d9-40d6-844a-4525d3733f98\" (UID: \"b13e2930-e5d9-40d6-844a-4525d3733f98\") "
Jul 28 05:07:56.053528 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:56.053507    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b13e2930-e5d9-40d6-844a-4525d3733f98-kube-api-access-49z2v" (OuterVolumeSpecName: "kube-api-access-49z2v") pod "b13e2930-e5d9-40d6-844a-4525d3733f98" (UID: "b13e2930-e5d9-40d6-844a-4525d3733f98"). InnerVolumeSpecName "kube-api-access-49z2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 05:07:56.153032 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:56.153003    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49z2v\" (UniqueName: \"kubernetes.io/projected/b13e2930-e5d9-40d6-844a-4525d3733f98-kube-api-access-49z2v\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:07:56.748175 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:56.748146    2574 generic.go:356] "Generic (PLEG): container finished" podID="b13e2930-e5d9-40d6-844a-4525d3733f98" containerID="be475e92391cb03988fa411aec321cd40625ccf532adb3a98082842749961a3d" exitCode=0
Jul 28 05:07:56.748545 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:56.748187    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr"
Jul 28 05:07:56.748545 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:56.748235    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr" event={"ID":"b13e2930-e5d9-40d6-844a-4525d3733f98","Type":"ContainerDied","Data":"be475e92391cb03988fa411aec321cd40625ccf532adb3a98082842749961a3d"}
Jul 28 05:07:56.748545 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:56.748280    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vwcdr" event={"ID":"b13e2930-e5d9-40d6-844a-4525d3733f98","Type":"ContainerDied","Data":"ad8a81686eb9147204db53e338c5398c1e5f21ba0332fe5b0cbdb4f0eba660e0"}
Jul 28 05:07:56.748545 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:56.748305    2574 scope.go:117] "RemoveContainer" containerID="be475e92391cb03988fa411aec321cd40625ccf532adb3a98082842749961a3d"
Jul 28 05:07:56.757875 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:56.757861    2574 scope.go:117] "RemoveContainer" containerID="be475e92391cb03988fa411aec321cd40625ccf532adb3a98082842749961a3d"
Jul 28 05:07:56.758105 ip-10-0-133-204 kubenswrapper[2574]: E0728 05:07:56.758085    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be475e92391cb03988fa411aec321cd40625ccf532adb3a98082842749961a3d\": container with ID starting with be475e92391cb03988fa411aec321cd40625ccf532adb3a98082842749961a3d not found: ID does not exist" containerID="be475e92391cb03988fa411aec321cd40625ccf532adb3a98082842749961a3d"
Jul 28 05:07:56.758156 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:56.758111    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be475e92391cb03988fa411aec321cd40625ccf532adb3a98082842749961a3d"} err="failed to get container status \"be475e92391cb03988fa411aec321cd40625ccf532adb3a98082842749961a3d\": rpc error: code = NotFound desc = could not find container \"be475e92391cb03988fa411aec321cd40625ccf532adb3a98082842749961a3d\": container with ID starting with be475e92391cb03988fa411aec321cd40625ccf532adb3a98082842749961a3d not found: ID does not exist"
Jul 28 05:07:56.768042 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:56.768014    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vwcdr"]
Jul 28 05:07:56.773501 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:56.773480    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vwcdr"]
Jul 28 05:07:57.350680 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:57.350649    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:57.451293 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:57.451262    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:57.505607 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:57.505582    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:07:57.620334 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:57.620263    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b13e2930-e5d9-40d6-844a-4525d3733f98" path="/var/lib/kubelet/pods/b13e2930-e5d9-40d6-844a-4525d3733f98/volumes"
Jul 28 05:07:58.575408 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:07:58.575376    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:08:01.918503 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:01.918472    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:08:01.923451 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:01.923425    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k"]
Jul 28 05:08:01.926595 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:01.926573    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-dc1efb-5nn2k"]
Jul 28 05:08:02.344959 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:02.344928    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:08:03.619943 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:03.619912    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbc508d1-f541-41d7-bf6f-69038dcf56b3" path="/var/lib/kubelet/pods/dbc508d1-f541-41d7-bf6f-69038dcf56b3/volumes"
Jul 28 05:08:07.328275 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:07.328240    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"]
Jul 28 05:08:07.372473 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:07.372435    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:08:09.178010 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:09.177975    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:08:22.347626 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.347574    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l" podUID="641a25c9-f006-49fb-8b9b-527d0975dc82" containerName="main" containerID="cri-o://9fd07612511ab4685142b23bc291cda062dfa81099b74994ab6d9239369297f8" gracePeriod=45
Jul 28 05:08:22.601392 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.601322    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:08:22.768897 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.768863    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-tmp-dir\") pod \"641a25c9-f006-49fb-8b9b-527d0975dc82\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") "
Jul 28 05:08:22.769079 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.768957    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-home\") pod \"641a25c9-f006-49fb-8b9b-527d0975dc82\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") "
Jul 28 05:08:22.769079 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.768997    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2t9r\" (UniqueName: \"kubernetes.io/projected/641a25c9-f006-49fb-8b9b-527d0975dc82-kube-api-access-z2t9r\") pod \"641a25c9-f006-49fb-8b9b-527d0975dc82\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") "
Jul 28 05:08:22.769079 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.769026    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/641a25c9-f006-49fb-8b9b-527d0975dc82-tls-certs\") pod \"641a25c9-f006-49fb-8b9b-527d0975dc82\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") "
Jul 28 05:08:22.769262 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.769081    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-dshm\") pod \"641a25c9-f006-49fb-8b9b-527d0975dc82\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") "
Jul 28 05:08:22.769262 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.769107    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-model-cache\") pod \"641a25c9-f006-49fb-8b9b-527d0975dc82\" (UID: \"641a25c9-f006-49fb-8b9b-527d0975dc82\") "
Jul 28 05:08:22.769262 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.769154    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "641a25c9-f006-49fb-8b9b-527d0975dc82" (UID: "641a25c9-f006-49fb-8b9b-527d0975dc82"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:08:22.769450 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.769259    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-home" (OuterVolumeSpecName: "home") pod "641a25c9-f006-49fb-8b9b-527d0975dc82" (UID: "641a25c9-f006-49fb-8b9b-527d0975dc82"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:08:22.769450 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.769439    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-tmp-dir\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:08:22.769570 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.769462    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-home\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:08:22.769626 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.769561    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-model-cache" (OuterVolumeSpecName: "model-cache") pod "641a25c9-f006-49fb-8b9b-527d0975dc82" (UID: "641a25c9-f006-49fb-8b9b-527d0975dc82"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:08:22.771317 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.771269    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/641a25c9-f006-49fb-8b9b-527d0975dc82-kube-api-access-z2t9r" (OuterVolumeSpecName: "kube-api-access-z2t9r") pod "641a25c9-f006-49fb-8b9b-527d0975dc82" (UID: "641a25c9-f006-49fb-8b9b-527d0975dc82"). InnerVolumeSpecName "kube-api-access-z2t9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 05:08:22.771623 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.771603    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-dshm" (OuterVolumeSpecName: "dshm") pod "641a25c9-f006-49fb-8b9b-527d0975dc82" (UID: "641a25c9-f006-49fb-8b9b-527d0975dc82"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 05:08:22.771702 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.771618    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/641a25c9-f006-49fb-8b9b-527d0975dc82-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "641a25c9-f006-49fb-8b9b-527d0975dc82" (UID: "641a25c9-f006-49fb-8b9b-527d0975dc82"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 05:08:22.870555 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.870504    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-dshm\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:08:22.870555 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.870522    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/641a25c9-f006-49fb-8b9b-527d0975dc82-model-cache\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:08:22.870555 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.870533    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2t9r\" (UniqueName: \"kubernetes.io/projected/641a25c9-f006-49fb-8b9b-527d0975dc82-kube-api-access-z2t9r\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:08:22.870555 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.870541    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/641a25c9-f006-49fb-8b9b-527d0975dc82-tls-certs\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:08:22.874585 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.874562    2574 generic.go:356] "Generic (PLEG): container finished" podID="641a25c9-f006-49fb-8b9b-527d0975dc82" containerID="9fd07612511ab4685142b23bc291cda062dfa81099b74994ab6d9239369297f8" exitCode=0
Jul 28 05:08:22.874674 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.874603    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"
Jul 28 05:08:22.874674 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.874645    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l" event={"ID":"641a25c9-f006-49fb-8b9b-527d0975dc82","Type":"ContainerDied","Data":"9fd07612511ab4685142b23bc291cda062dfa81099b74994ab6d9239369297f8"}
Jul 28 05:08:22.874788 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.874690    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l" event={"ID":"641a25c9-f006-49fb-8b9b-527d0975dc82","Type":"ContainerDied","Data":"4ba7a5de849a4cfc9e18c6fcff5d1eb341f8f9262e03c0f474d97682290b19b1"}
Jul 28 05:08:22.874788 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.874715    2574 scope.go:117] "RemoveContainer" containerID="9fd07612511ab4685142b23bc291cda062dfa81099b74994ab6d9239369297f8"
Jul 28 05:08:22.883122 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.883106    2574 scope.go:117] "RemoveContainer" containerID="9fd07612511ab4685142b23bc291cda062dfa81099b74994ab6d9239369297f8"
Jul 28 05:08:22.883393 ip-10-0-133-204 kubenswrapper[2574]: E0728 05:08:22.883354    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fd07612511ab4685142b23bc291cda062dfa81099b74994ab6d9239369297f8\": container with ID starting with 9fd07612511ab4685142b23bc291cda062dfa81099b74994ab6d9239369297f8 not found: ID does not exist" containerID="9fd07612511ab4685142b23bc291cda062dfa81099b74994ab6d9239369297f8"
Jul 28 05:08:22.883457 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.883404    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fd07612511ab4685142b23bc291cda062dfa81099b74994ab6d9239369297f8"} err="failed to get container status \"9fd07612511ab4685142b23bc291cda062dfa81099b74994ab6d9239369297f8\": rpc error: code = NotFound desc = could not find container \"9fd07612511ab4685142b23bc291cda062dfa81099b74994ab6d9239369297f8\": container with ID starting with 9fd07612511ab4685142b23bc291cda062dfa81099b74994ab6d9239369297f8 not found: ID does not exist"
Jul 28 05:08:22.898306 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.898281    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"]
Jul 28 05:08:22.899858 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:22.899839    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-dc1efb/ipp-route-dc1efb-kserve-6f78f56c9f-qcb5l"]
Jul 28 05:08:23.620702 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:23.620672    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="641a25c9-f006-49fb-8b9b-527d0975dc82" path="/var/lib/kubelet/pods/641a25c9-f006-49fb-8b9b-527d0975dc82/volumes"
Jul 28 05:08:43.532537 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:43.532454    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:08:44.185405 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:44.185350    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:08:49.667812 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:49.667762    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:08:49.731672 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:49.731646    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:08:49.977137 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:49.977070    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:08:50.231948 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.231876    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:08:50.379603 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.379575    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g"]
Jul 28 05:08:50.379927 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.379915    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbc508d1-f541-41d7-bf6f-69038dcf56b3" containerName="revoke-keys"
Jul 28 05:08:50.379978 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.379928    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbc508d1-f541-41d7-bf6f-69038dcf56b3" containerName="revoke-keys"
Jul 28 05:08:50.379978 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.379938    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b13e2930-e5d9-40d6-844a-4525d3733f98" containerName="registry-server"
Jul 28 05:08:50.379978 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.379945    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b13e2930-e5d9-40d6-844a-4525d3733f98" containerName="registry-server"
Jul 28 05:08:50.379978 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.379960    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="641a25c9-f006-49fb-8b9b-527d0975dc82" containerName="main"
Jul 28 05:08:50.379978 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.379965    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="641a25c9-f006-49fb-8b9b-527d0975dc82" containerName="main"
Jul 28 05:08:50.380140 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.380027    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b13e2930-e5d9-40d6-844a-4525d3733f98" containerName="registry-server"
Jul 28 05:08:50.380140 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.380036    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbc508d1-f541-41d7-bf6f-69038dcf56b3" containerName="revoke-keys"
Jul 28 05:08:50.380140 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.380045    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="641a25c9-f006-49fb-8b9b-527d0975dc82" containerName="main"
Jul 28 05:08:50.384580 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.384561    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g"
Jul 28 05:08:50.386569 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.386550    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 05:08:50.386681 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.386573    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 05:08:50.392282 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.392253    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g"]
Jul 28 05:08:50.496386 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.496302    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f725aa-2e37-41df-8798-cc599cb60857-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g\" (UID: \"01f725aa-2e37-41df-8798-cc599cb60857\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g"
Jul 28 05:08:50.597180 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.597157    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f725aa-2e37-41df-8798-cc599cb60857-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g\" (UID: \"01f725aa-2e37-41df-8798-cc599cb60857\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g"
Jul 28 05:08:50.597778 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.597762    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f725aa-2e37-41df-8798-cc599cb60857-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g\" (UID: \"01f725aa-2e37-41df-8798-cc599cb60857\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g"
Jul 28 05:08:50.695786 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.695763    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g"
Jul 28 05:08:50.828951 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:50.828923    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g"]
Jul 28 05:08:50.830931 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:08:50.830905    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01f725aa_2e37_41df_8798_cc599cb60857.slice/crio-79afd80a038b22a96478aa57791f8ce98114cd99831cf77c11b928945f38fdd0 WatchSource:0}: Error finding container 79afd80a038b22a96478aa57791f8ce98114cd99831cf77c11b928945f38fdd0: Status 404 returned error can't find the container with id 79afd80a038b22a96478aa57791f8ce98114cd99831cf77c11b928945f38fdd0
Jul 28 05:08:51.018458 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:51.018431    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g" event={"ID":"01f725aa-2e37-41df-8798-cc599cb60857","Type":"ContainerStarted","Data":"79afd80a038b22a96478aa57791f8ce98114cd99831cf77c11b928945f38fdd0"}
Jul 28 05:08:52.024991 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:52.024913    2574 generic.go:356] "Generic (PLEG): container finished" podID="01f725aa-2e37-41df-8798-cc599cb60857" containerID="a3898c34349fe7b9dea1eb2e70567c26dffbd3980b0bb8146a8cdfd0f3eafe14" exitCode=0
Jul 28 05:08:52.025330 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:52.025002    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g" event={"ID":"01f725aa-2e37-41df-8798-cc599cb60857","Type":"ContainerDied","Data":"a3898c34349fe7b9dea1eb2e70567c26dffbd3980b0bb8146a8cdfd0f3eafe14"}
Jul 28 05:08:53.153154 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:53.153125    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g"
Jul 28 05:08:53.218544 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:53.218517    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f725aa-2e37-41df-8798-cc599cb60857-maas-api-service-ca\") pod \"01f725aa-2e37-41df-8798-cc599cb60857\" (UID: \"01f725aa-2e37-41df-8798-cc599cb60857\") "
Jul 28 05:08:53.218834 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:53.218814    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01f725aa-2e37-41df-8798-cc599cb60857-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01f725aa-2e37-41df-8798-cc599cb60857" (UID: "01f725aa-2e37-41df-8798-cc599cb60857"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 05:08:53.319108 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:53.319056    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f725aa-2e37-41df-8798-cc599cb60857-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:08:54.035334 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:54.035309    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g"
Jul 28 05:08:54.035485 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:54.035336    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g" event={"ID":"01f725aa-2e37-41df-8798-cc599cb60857","Type":"ContainerDied","Data":"79afd80a038b22a96478aa57791f8ce98114cd99831cf77c11b928945f38fdd0"}
Jul 28 05:08:54.035485 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:08:54.035424    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79afd80a038b22a96478aa57791f8ce98114cd99831cf77c11b928945f38fdd0"
Jul 28 05:09:00.912720 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:00.912690    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:01.010386 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:01.010339    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:01.149338 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:01.149302    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:04.281641 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:04.281611    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:09:05.443976 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:05.443943    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g"]
Jul 28 05:09:05.445661 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:05.445639    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9b749c-pl76g"]
Jul 28 05:09:05.448388 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:05.448350    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:05.620021 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:05.619998    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01f725aa-2e37-41df-8798-cc599cb60857" path="/var/lib/kubelet/pods/01f725aa-2e37-41df-8798-cc599cb60857/volumes"
Jul 28 05:09:05.852872 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:05.852835    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:06.429783 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:06.429754    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:11.575271 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:11.575232    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:11.629486 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:11.629454    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:11.857466 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:11.857392    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:12.106059 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.106025    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:12.256731 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.256658    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf"]
Jul 28 05:09:12.257040 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.257028    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01f725aa-2e37-41df-8798-cc599cb60857" containerName="revoke-keys"
Jul 28 05:09:12.257086 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.257042    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="01f725aa-2e37-41df-8798-cc599cb60857" containerName="revoke-keys"
Jul 28 05:09:12.257120 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.257099    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="01f725aa-2e37-41df-8798-cc599cb60857" containerName="revoke-keys"
Jul 28 05:09:12.260204 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.260188    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf"
Jul 28 05:09:12.262289 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.262256    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 05:09:12.262450 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.262434    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 05:09:12.269838 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.269816    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf"]
Jul 28 05:09:12.366484 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.366452    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51112a51-ca8d-4296-bf31-6b4de23559ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf\" (UID: \"51112a51-ca8d-4296-bf31-6b4de23559ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf"
Jul 28 05:09:12.467163 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.467113    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51112a51-ca8d-4296-bf31-6b4de23559ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf\" (UID: \"51112a51-ca8d-4296-bf31-6b4de23559ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf"
Jul 28 05:09:12.467759 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.467741    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51112a51-ca8d-4296-bf31-6b4de23559ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf\" (UID: \"51112a51-ca8d-4296-bf31-6b4de23559ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf"
Jul 28 05:09:12.571192 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.571164    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf"
Jul 28 05:09:12.702028 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:12.702002    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf"]
Jul 28 05:09:12.704220 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:09:12.704191    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51112a51_ca8d_4296_bf31_6b4de23559ef.slice/crio-d8ad67cac51d5003648640f638c1d68456ec647beb90613d9fb2b1254ae54a40 WatchSource:0}: Error finding container d8ad67cac51d5003648640f638c1d68456ec647beb90613d9fb2b1254ae54a40: Status 404 returned error can't find the container with id d8ad67cac51d5003648640f638c1d68456ec647beb90613d9fb2b1254ae54a40
Jul 28 05:09:13.128416 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:13.128306    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf" event={"ID":"51112a51-ca8d-4296-bf31-6b4de23559ef","Type":"ContainerStarted","Data":"aa92663523214321a31e141f3f327ed6f4c8873b7b8bcdeadffc37124035af52"}
Jul 28 05:09:13.128416 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:13.128386    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf" event={"ID":"51112a51-ca8d-4296-bf31-6b4de23559ef","Type":"ContainerStarted","Data":"d8ad67cac51d5003648640f638c1d68456ec647beb90613d9fb2b1254ae54a40"}
Jul 28 05:09:13.143424 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:13.143376    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf" podStartSLOduration=1.143343163 podStartE2EDuration="1.143343163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 05:09:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 05:09:13.141647315 +0000 UTC m=+3448.083487036" watchObservedRunningTime="2026-07-28 05:09:13.143343163 +0000 UTC m=+3448.085182909"
Jul 28 05:09:14.134310 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:14.134276    2574 generic.go:356] "Generic (PLEG): container finished" podID="51112a51-ca8d-4296-bf31-6b4de23559ef" containerID="aa92663523214321a31e141f3f327ed6f4c8873b7b8bcdeadffc37124035af52" exitCode=0
Jul 28 05:09:14.134731 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:14.134378    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf" event={"ID":"51112a51-ca8d-4296-bf31-6b4de23559ef","Type":"ContainerDied","Data":"aa92663523214321a31e141f3f327ed6f4c8873b7b8bcdeadffc37124035af52"}
Jul 28 05:09:15.268904 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:15.268876    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf"
Jul 28 05:09:15.390821 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:15.390735    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51112a51-ca8d-4296-bf31-6b4de23559ef-maas-api-service-ca\") pod \"51112a51-ca8d-4296-bf31-6b4de23559ef\" (UID: \"51112a51-ca8d-4296-bf31-6b4de23559ef\") "
Jul 28 05:09:15.391107 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:15.391084    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51112a51-ca8d-4296-bf31-6b4de23559ef-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "51112a51-ca8d-4296-bf31-6b4de23559ef" (UID: "51112a51-ca8d-4296-bf31-6b4de23559ef"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 05:09:15.491419 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:15.491381    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51112a51-ca8d-4296-bf31-6b4de23559ef-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:09:16.144835 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:16.144807    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf"
Jul 28 05:09:16.144993 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:16.144801    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf" event={"ID":"51112a51-ca8d-4296-bf31-6b4de23559ef","Type":"ContainerDied","Data":"d8ad67cac51d5003648640f638c1d68456ec647beb90613d9fb2b1254ae54a40"}
Jul 28 05:09:16.144993 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:16.144932    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d8ad67cac51d5003648640f638c1d68456ec647beb90613d9fb2b1254ae54a40"
Jul 28 05:09:22.840270 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:22.840233    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf"]
Jul 28 05:09:22.841920 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:22.841901    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1badbb-29753580-45cdf"]
Jul 28 05:09:22.846922 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:22.846901    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:22.911979 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:22.911948    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:23.012025 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:23.011998    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:23.620441 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:23.620411    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce95dfe8-f309-44b8-987e-454890e68b7b" path="/var/lib/kubelet/pods/ce95dfe8-f309-44b8-987e-454890e68b7b/volumes"
Jul 28 05:09:26.271131 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:26.271103    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:09:27.335399 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:27.335349    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:27.336655 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:27.336629    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf"]
Jul 28 05:09:27.338400 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:27.338381    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1badbb-f58mf"]
Jul 28 05:09:27.619809 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:27.619749    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51112a51-ca8d-4296-bf31-6b4de23559ef" path="/var/lib/kubelet/pods/51112a51-ca8d-4296-bf31-6b4de23559ef/volumes"
Jul 28 05:09:27.741633 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:27.741611    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:33.424897 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:33.424867    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:33.525829 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:33.525789    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:33.697415 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:33.697331    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:33.934328 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:33.934297    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:34.056972 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:34.056941    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b"]
Jul 28 05:09:34.057390 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:34.057350    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51112a51-ca8d-4296-bf31-6b4de23559ef" containerName="revoke-keys"
Jul 28 05:09:34.057390 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:34.057382    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="51112a51-ca8d-4296-bf31-6b4de23559ef" containerName="revoke-keys"
Jul 28 05:09:34.057538 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:34.057490    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="51112a51-ca8d-4296-bf31-6b4de23559ef" containerName="revoke-keys"
Jul 28 05:09:34.060500 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:34.060482    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b"
Jul 28 05:09:34.062431 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:34.062413    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 05:09:34.062505 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:34.062417    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-874h6\""
Jul 28 05:09:34.066751 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:34.066730    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b"]
Jul 28 05:09:34.226626 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:34.221459    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17803d5f-e83a-4e1c-9bb6-a2253af41566-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b\" (UID: \"17803d5f-e83a-4e1c-9bb6-a2253af41566\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b"
Jul 28 05:09:34.322832 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:34.322758    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17803d5f-e83a-4e1c-9bb6-a2253af41566-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b\" (UID: \"17803d5f-e83a-4e1c-9bb6-a2253af41566\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b"
Jul 28 05:09:34.323305 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:34.323288    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17803d5f-e83a-4e1c-9bb6-a2253af41566-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b\" (UID: \"17803d5f-e83a-4e1c-9bb6-a2253af41566\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b"
Jul 28 05:09:34.372026 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:34.372002    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b"
Jul 28 05:09:34.496950 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:34.496927    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b"]
Jul 28 05:09:34.498958 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:09:34.498931    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17803d5f_e83a_4e1c_9bb6_a2253af41566.slice/crio-adaa19457223d1410dc54bd999594567c5a2820e582d0fafb3b083920e789880 WatchSource:0}: Error finding container adaa19457223d1410dc54bd999594567c5a2820e582d0fafb3b083920e789880: Status 404 returned error can't find the container with id adaa19457223d1410dc54bd999594567c5a2820e582d0fafb3b083920e789880
Jul 28 05:09:35.240174 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:35.240148    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b" event={"ID":"17803d5f-e83a-4e1c-9bb6-a2253af41566","Type":"ContainerStarted","Data":"6cc5c0f2d6e6db0f37303fe6c54c13e70dc0e93412982034ac402072871fc759"}
Jul 28 05:09:35.240304 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:35.240196    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b" event={"ID":"17803d5f-e83a-4e1c-9bb6-a2253af41566","Type":"ContainerStarted","Data":"adaa19457223d1410dc54bd999594567c5a2820e582d0fafb3b083920e789880"}
Jul 28 05:09:35.255121 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:35.255080    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b" podStartSLOduration=1.255065728 podStartE2EDuration="1.255065728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 05:09:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 05:09:35.252513161 +0000 UTC m=+3470.194352882" watchObservedRunningTime="2026-07-28 05:09:35.255065728 +0000 UTC m=+3470.196905447"
Jul 28 05:09:36.246484 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:36.246451    2574 generic.go:356] "Generic (PLEG): container finished" podID="17803d5f-e83a-4e1c-9bb6-a2253af41566" containerID="6cc5c0f2d6e6db0f37303fe6c54c13e70dc0e93412982034ac402072871fc759" exitCode=0
Jul 28 05:09:36.246836 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:36.246506    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b" event={"ID":"17803d5f-e83a-4e1c-9bb6-a2253af41566","Type":"ContainerDied","Data":"6cc5c0f2d6e6db0f37303fe6c54c13e70dc0e93412982034ac402072871fc759"}
Jul 28 05:09:37.376522 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:37.376495    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b"
Jul 28 05:09:37.548634 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:37.548607    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17803d5f-e83a-4e1c-9bb6-a2253af41566-maas-api-service-ca\") pod \"17803d5f-e83a-4e1c-9bb6-a2253af41566\" (UID: \"17803d5f-e83a-4e1c-9bb6-a2253af41566\") "
Jul 28 05:09:37.548956 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:37.548932    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17803d5f-e83a-4e1c-9bb6-a2253af41566-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "17803d5f-e83a-4e1c-9bb6-a2253af41566" (UID: "17803d5f-e83a-4e1c-9bb6-a2253af41566"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 05:09:37.649943 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:37.649922    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17803d5f-e83a-4e1c-9bb6-a2253af41566-maas-api-service-ca\") on node \"ip-10-0-133-204.ec2.internal\" DevicePath \"\""
Jul 28 05:09:38.256530 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:38.256505    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b"
Jul 28 05:09:38.256667 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:38.256546    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b" event={"ID":"17803d5f-e83a-4e1c-9bb6-a2253af41566","Type":"ContainerDied","Data":"adaa19457223d1410dc54bd999594567c5a2820e582d0fafb3b083920e789880"}
Jul 28 05:09:38.256667 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:38.256587    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="adaa19457223d1410dc54bd999594567c5a2820e582d0fafb3b083920e789880"
Jul 28 05:09:44.594013 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:44.593978    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:44.594424 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:44.594134    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v"]
Jul 28 05:09:44.596767 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:44.596734    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-92c557-29753580-s864v"]
Jul 28 05:09:44.692428 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:44.692399    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:44.791446 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:44.791421    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:45.620623 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:45.620593    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7063fed-a260-4312-860d-54529267118b" path="/var/lib/kubelet/pods/c7063fed-a260-4312-860d-54529267118b/volumes"
Jul 28 05:09:45.964312 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:45.964234    2574 scope.go:117] "RemoveContainer" containerID="0687f09bd0d222a1b2f9b805b968b267ed8b54ca244ef4cd2ea43634addd4a7e"
Jul 28 05:09:45.972447 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:45.972428    2574 scope.go:117] "RemoveContainer" containerID="ddb1e5fb669bfc8779d344055846bed188c8a43463de4b41b71a213c5ceb3be4"
Jul 28 05:09:47.577723 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:47.577683    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sp9fh"]
Jul 28 05:09:49.139400 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:49.139248    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b"]
Jul 28 05:09:49.139865 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:49.139443    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:49.140957 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:49.140935    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-92c557-2lb6b"]
Jul 28 05:09:49.550734 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:49.550708    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:49.620355 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:49.620316    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17803d5f-e83a-4e1c-9bb6-a2253af41566" path="/var/lib/kubelet/pods/17803d5f-e83a-4e1c-9bb6-a2253af41566/volumes"
Jul 28 05:09:49.689115 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:49.689083    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:55.302163 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:55.302127    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:55.362845 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:55.362809    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:55.577882 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:55.577804    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:09:55.828489 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:09:55.828408    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:10:01.952289 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:01.952255    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vkgnk_9407f103-7ffb-4eb6-8bf2-7abd5c801b80/manager/0.log"
Jul 28 05:10:02.082563 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:02.082531    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zln4n_01bd8eb2-f8d0-4b6a-9709-36b19d94bf85/manager/0.log"
Jul 28 05:10:02.336245 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:02.336219    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5zmc7_4df97ea0-3f6e-467c-b450-6d795d40119d/server/0.log"
Jul 28 05:10:02.461380 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:02.461327    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pgzfs_4b3ebb7d-856f-4e16-9e04-786a216dee39/manager/0.log"
Jul 28 05:10:02.715663 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:02.715585    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69d65f9dd8-kd8fw_f5167944-00ac-41a7-9ac6-007f32e9d34c/copy-manifests/0.log"
Jul 28 05:10:02.727444 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:02.727418    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69d65f9dd8-kd8fw_f5167944-00ac-41a7-9ac6-007f32e9d34c/manager/0.log"
Jul 28 05:10:03.472769 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:03.472742    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29753565-svt8q_14280d8e-235e-429b-9785-ed2ff2b5fcf5/cleanup/0.log"
Jul 28 05:10:03.591680 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:03.591655    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29753580-tq6qw_03f427be-6c0a-4363-9332-5bca149df170/cleanup/0.log"
Jul 28 05:10:05.839788 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:05.839761    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sp9fh_c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0/limitador/0.log"
Jul 28 05:10:05.967870 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:05.967833    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lps4s_628810bb-4c2f-44d7-8be0-42d1a8fdb8ef/manager/0.log"
Jul 28 05:10:06.481589 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:06.481559    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zr6b6_d1c446cc-aab6-4231-b1bf-eb763e839406/discovery/0.log"
Jul 28 05:10:06.729930 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:06.729904    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58d69b594d-z46js_ebe57051-0cab-4985-82eb-a0f619d1b35f/kube-auth-proxy/0.log"
Jul 28 05:10:07.972957 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:07.972879    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9tfcx_39734416-47e7-436e-a8da-fb8bd981d198/main/0.log"
Jul 28 05:10:14.259654 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:14.259623    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:10:14.359968 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:14.359927    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log" limit=1024
Jul 28 05:10:16.913739 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:16.913693    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wgdm4_95552c94-10a3-4446-8b07-fc7afaadc0d4/global-pull-secret-syncer/0.log"
Jul 28 05:10:16.994714 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:16.994684    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b47jl_ad63d3ab-011e-4dd2-ba86-f57c268eefcd/konnectivity-agent/0.log"
Jul 28 05:10:17.122014 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:17.121981    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-204.ec2.internal_6db9e0d726c4428a7173af2b853a8bd3/haproxy/0.log"
Jul 28 05:10:22.507547 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:22.507519    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sp9fh_c808b49a-2a89-4218-a1d4-4a8ea1a5f4d0/limitador/0.log"
Jul 28 05:10:22.577959 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:22.577924    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lps4s_628810bb-4c2f-44d7-8be0-42d1a8fdb8ef/manager/0.log"
Jul 28 05:10:24.278409 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.278355    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_559a3031-9899-489c-b0b6-2012a7e14de5/alertmanager/0.log"
Jul 28 05:10:24.304351 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.304324    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_559a3031-9899-489c-b0b6-2012a7e14de5/config-reloader/0.log"
Jul 28 05:10:24.329594 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.329569    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_559a3031-9899-489c-b0b6-2012a7e14de5/kube-rbac-proxy-web/0.log"
Jul 28 05:10:24.352546 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.352514    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_559a3031-9899-489c-b0b6-2012a7e14de5/kube-rbac-proxy/0.log"
Jul 28 05:10:24.374047 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.374019    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_559a3031-9899-489c-b0b6-2012a7e14de5/kube-rbac-proxy-metric/0.log"
Jul 28 05:10:24.393958 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.393934    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_559a3031-9899-489c-b0b6-2012a7e14de5/prom-label-proxy/0.log"
Jul 28 05:10:24.413158 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.413125    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_559a3031-9899-489c-b0b6-2012a7e14de5/init-config-reloader/0.log"
Jul 28 05:10:24.585989 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.585884    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-c7fcv_9209391d-eaf3-4e2b-b501-e981beb2c808/monitoring-plugin/0.log"
Jul 28 05:10:24.784530 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.784488    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d5m5q_2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc/node-exporter/0.log"
Jul 28 05:10:24.805298 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.805271    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d5m5q_2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc/kube-rbac-proxy/0.log"
Jul 28 05:10:24.830208 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.830181    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d5m5q_2a67d072-1cde-43ed-bb01-dfa4ff6f0dcc/init-textfile/0.log"
Jul 28 05:10:24.857385 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.857305    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-pkpjb_478a0703-2277-4856-aead-a0b3e7a01fc9/kube-rbac-proxy-main/0.log"
Jul 28 05:10:24.883070 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.883034    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-pkpjb_478a0703-2277-4856-aead-a0b3e7a01fc9/kube-rbac-proxy-self/0.log"
Jul 28 05:10:24.903354 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:24.903325    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-pkpjb_478a0703-2277-4856-aead-a0b3e7a01fc9/openshift-state-metrics/0.log"
Jul 28 05:10:25.223966 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:25.223891    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55cc884f86-9rwmx_64a49d09-7679-4108-a2b3-2645497e4695/telemeter-client/0.log"
Jul 28 05:10:25.248219 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:25.248191    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55cc884f86-9rwmx_64a49d09-7679-4108-a2b3-2645497e4695/reload/0.log"
Jul 28 05:10:25.269268 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:25.269243    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55cc884f86-9rwmx_64a49d09-7679-4108-a2b3-2645497e4695/kube-rbac-proxy/0.log"
Jul 28 05:10:25.298025 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:25.297994    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-654777ddcb-bxmmw_6c12900c-a219-4f38-9b10-774b733f73b8/thanos-query/0.log"
Jul 28 05:10:25.319405 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:25.319356    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-654777ddcb-bxmmw_6c12900c-a219-4f38-9b10-774b733f73b8/kube-rbac-proxy-web/0.log"
Jul 28 05:10:25.339922 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:25.339898    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-654777ddcb-bxmmw_6c12900c-a219-4f38-9b10-774b733f73b8/kube-rbac-proxy/0.log"
Jul 28 05:10:25.359104 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:25.359085    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-654777ddcb-bxmmw_6c12900c-a219-4f38-9b10-774b733f73b8/prom-label-proxy/0.log"
Jul 28 05:10:25.382001 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:25.381962    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-654777ddcb-bxmmw_6c12900c-a219-4f38-9b10-774b733f73b8/kube-rbac-proxy-rules/0.log"
Jul 28 05:10:25.401516 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:25.401493    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-654777ddcb-bxmmw_6c12900c-a219-4f38-9b10-774b733f73b8/kube-rbac-proxy-metrics/0.log"
Jul 28 05:10:27.256563 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.256520    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"]
Jul 28 05:10:27.257078 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.257056    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17803d5f-e83a-4e1c-9bb6-a2253af41566" containerName="revoke-keys"
Jul 28 05:10:27.257159 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.257081    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="17803d5f-e83a-4e1c-9bb6-a2253af41566" containerName="revoke-keys"
Jul 28 05:10:27.257214 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.257195    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="17803d5f-e83a-4e1c-9bb6-a2253af41566" containerName="revoke-keys"
Jul 28 05:10:27.260408 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.260387    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.262097 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.262073    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5wtbr\"/\"kube-root-ca.crt\""
Jul 28 05:10:27.262742 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.262723    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5wtbr\"/\"default-dockercfg-8sxlx\""
Jul 28 05:10:27.262897 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.262730    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5wtbr\"/\"openshift-service-ca.crt\""
Jul 28 05:10:27.268293 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.268271    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"]
Jul 28 05:10:27.347875 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.347829    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/69980d45-740f-4c3b-8023-639fe6b60f17-proc\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.347875 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.347891    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtmsz\" (UniqueName: \"kubernetes.io/projected/69980d45-740f-4c3b-8023-639fe6b60f17-kube-api-access-qtmsz\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.348106 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.347949    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69980d45-740f-4c3b-8023-639fe6b60f17-sys\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.348106 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.347970    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69980d45-740f-4c3b-8023-639fe6b60f17-lib-modules\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.348106 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.348027    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/69980d45-740f-4c3b-8023-639fe6b60f17-podres\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.449549 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.449513    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/69980d45-740f-4c3b-8023-639fe6b60f17-proc\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.449549 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.449555    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtmsz\" (UniqueName: \"kubernetes.io/projected/69980d45-740f-4c3b-8023-639fe6b60f17-kube-api-access-qtmsz\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.449750 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.449583    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69980d45-740f-4c3b-8023-639fe6b60f17-sys\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.449750 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.449600    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69980d45-740f-4c3b-8023-639fe6b60f17-lib-modules\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.449750 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.449648    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/69980d45-740f-4c3b-8023-639fe6b60f17-proc\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.449750 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.449665    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69980d45-740f-4c3b-8023-639fe6b60f17-sys\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.449750 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.449672    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/69980d45-740f-4c3b-8023-639fe6b60f17-podres\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.449917 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.449767    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69980d45-740f-4c3b-8023-639fe6b60f17-lib-modules\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.449917 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.449773    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/69980d45-740f-4c3b-8023-639fe6b60f17-podres\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.456367 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.456339    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtmsz\" (UniqueName: \"kubernetes.io/projected/69980d45-740f-4c3b-8023-639fe6b60f17-kube-api-access-qtmsz\") pod \"perf-node-gather-daemonset-cq7jd\" (UID: \"69980d45-740f-4c3b-8023-639fe6b60f17\") " pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.571669 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.571631    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:27.701444 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:27.701420    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"]
Jul 28 05:10:27.704012 ip-10-0-133-204 kubenswrapper[2574]: W0728 05:10:27.703982    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod69980d45_740f_4c3b_8023_639fe6b60f17.slice/crio-d6104a30c95b795c3dcc07a30717c95791aea436970dfd16c2ac5e229fc0d20e WatchSource:0}: Error finding container d6104a30c95b795c3dcc07a30717c95791aea436970dfd16c2ac5e229fc0d20e: Status 404 returned error can't find the container with id d6104a30c95b795c3dcc07a30717c95791aea436970dfd16c2ac5e229fc0d20e
Jul 28 05:10:28.501670 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:28.501634    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd" event={"ID":"69980d45-740f-4c3b-8023-639fe6b60f17","Type":"ContainerStarted","Data":"e977b9963e3bd4ec8a1e2b5eccc57a31c2fe19ba3734eddd437afd92fd910b8b"}
Jul 28 05:10:28.502053 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:28.501693    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd" event={"ID":"69980d45-740f-4c3b-8023-639fe6b60f17","Type":"ContainerStarted","Data":"d6104a30c95b795c3dcc07a30717c95791aea436970dfd16c2ac5e229fc0d20e"}
Jul 28 05:10:28.502053 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:28.501721    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:28.517035 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:28.516983    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd" podStartSLOduration=1.516966596 podStartE2EDuration="1.516966596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 05:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 05:10:28.515588269 +0000 UTC m=+3523.457427994" watchObservedRunningTime="2026-07-28 05:10:28.516966596 +0000 UTC m=+3523.458806366"
Jul 28 05:10:29.577811 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:29.577776    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vqdlc_3411355c-8caa-429b-88cc-97ddaabc7760/dns/0.log"
Jul 28 05:10:29.598311 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:29.598283    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vqdlc_3411355c-8caa-429b-88cc-97ddaabc7760/kube-rbac-proxy/0.log"
Jul 28 05:10:29.672176 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:29.672147    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xgf87_3ec166b8-6374-4e81-9a43-65a6fe177e7a/dns-node-resolver/0.log"
Jul 28 05:10:30.199334 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:30.199303    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-27l4c_d2385840-55b1-4920-aa89-ec64f34697f6/node-ca/0.log"
Jul 28 05:10:31.306432 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:31.306392    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zr6b6_d1c446cc-aab6-4231-b1bf-eb763e839406/discovery/0.log"
Jul 28 05:10:31.347888 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:31.347859    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58d69b594d-z46js_ebe57051-0cab-4985-82eb-a0f619d1b35f/kube-auth-proxy/0.log"
Jul 28 05:10:32.289033 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:32.289002    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hj7ns_a66b7189-fce5-4c9e-9722-188b5c351b16/serve-healthcheck-canary/0.log"
Jul 28 05:10:32.803089 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:32.803063    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-76frw_c5ff3174-cacb-4bc7-bb73-c6e89b47d427/kube-rbac-proxy/0.log"
Jul 28 05:10:32.822020 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:32.821996    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-76frw_c5ff3174-cacb-4bc7-bb73-c6e89b47d427/exporter/0.log"
Jul 28 05:10:32.841383 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:32.841341    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-76frw_c5ff3174-cacb-4bc7-bb73-c6e89b47d427/extractor/0.log"
Jul 28 05:10:34.516727 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:34.516699    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5wtbr/perf-node-gather-daemonset-cq7jd"
Jul 28 05:10:35.012064 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:35.012029    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vkgnk_9407f103-7ffb-4eb6-8bf2-7abd5c801b80/manager/0.log"
Jul 28 05:10:35.222831 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:35.222800    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zln4n_01bd8eb2-f8d0-4b6a-9709-36b19d94bf85/manager/0.log"
Jul 28 05:10:35.434604 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:35.434577    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5zmc7_4df97ea0-3f6e-467c-b450-6d795d40119d/server/0.log"
Jul 28 05:10:35.505200 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:35.505172    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pgzfs_4b3ebb7d-856f-4e16-9e04-786a216dee39/manager/0.log"
Jul 28 05:10:35.638100 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:35.638070    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69d65f9dd8-kd8fw_f5167944-00ac-41a7-9ac6-007f32e9d34c/manager/0.log"
Jul 28 05:10:35.668675 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:35.668650    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69d65f9dd8-kd8fw_f5167944-00ac-41a7-9ac6-007f32e9d34c/copy-manifests/0.log"
Jul 28 05:10:37.177343 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:37.177315    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-v74fx_067c7e82-47a0-4d63-8033-52625050f420/openshift-lws-operator/0.log"
Jul 28 05:10:37.855804 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:37.855770    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zr6b6_d1c446cc-aab6-4231-b1bf-eb763e839406/discovery/0.log"
Jul 28 05:10:37.881794 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:37.881770    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58d69b594d-z46js_ebe57051-0cab-4985-82eb-a0f619d1b35f/kube-auth-proxy/0.log"
Jul 28 05:10:39.479828 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:39.479785    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-zr6b6_d1c446cc-aab6-4231-b1bf-eb763e839406/discovery/0.log"
Jul 28 05:10:39.506058 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:39.506017    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58d69b594d-z46js_ebe57051-0cab-4985-82eb-a0f619d1b35f/kube-auth-proxy/0.log"
Jul 28 05:10:43.866419 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:43.866392    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrsls_679a6032-e93c-4030-b74b-6c5cdf46b470/kube-multus-additional-cni-plugins/0.log"
Jul 28 05:10:43.886075 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:43.886048    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrsls_679a6032-e93c-4030-b74b-6c5cdf46b470/egress-router-binary-copy/0.log"
Jul 28 05:10:43.909931 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:43.909906    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrsls_679a6032-e93c-4030-b74b-6c5cdf46b470/cni-plugins/0.log"
Jul 28 05:10:43.928703 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:43.928678    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrsls_679a6032-e93c-4030-b74b-6c5cdf46b470/bond-cni-plugin/0.log"
Jul 28 05:10:43.947346 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:43.947326    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrsls_679a6032-e93c-4030-b74b-6c5cdf46b470/routeoverride-cni/0.log"
Jul 28 05:10:43.969542 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:43.969524    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrsls_679a6032-e93c-4030-b74b-6c5cdf46b470/whereabouts-cni-bincopy/0.log"
Jul 28 05:10:43.988504 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:43.988479    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mrsls_679a6032-e93c-4030-b74b-6c5cdf46b470/whereabouts-cni/0.log"
Jul 28 05:10:44.062697 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:44.062679    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zl45v_5aaea95b-0d89-465d-a928-e55805bffaa8/kube-multus/0.log"
Jul 28 05:10:44.174412 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:44.174347    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-npqcf_ffa6e27a-0769-421d-a24d-3a9138bec37f/network-metrics-daemon/0.log"
Jul 28 05:10:44.193507 ip-10-0-133-204 kubenswrapper[2574]: I0728 05:10:44.193482    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-npqcf_ffa6e27a-0769-421d-a24d-3a9138bec37f/kube-rbac-proxy/0.log"