Jul 28 20:06:34.123207 ip-10-0-133-72 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 20:06:34.123215 ip-10-0-133-72 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 20:06:34.123222 ip-10-0-133-72 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 20:06:34.123458 ip-10-0-133-72 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 20:06:44.286408 ip-10-0-133-72 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 20:06:44.286428 ip-10-0-133-72 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 380584e75bae4d5e83b0475e2da848e0 --
Jul 28 20:08:42.645836 ip-10-0-133-72 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 20:08:43.032385 ip-10-0-133-72 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 20:08:43.032385 ip-10-0-133-72 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 20:08:43.032385 ip-10-0-133-72 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 20:08:43.032385 ip-10-0-133-72 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 20:08:43.032385 ip-10-0-133-72 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 20:08:43.033092 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.032429    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 20:08:43.036618 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036601    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:08:43.036618 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036617    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:08:43.036618 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036621    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036626    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036629    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036632    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036634    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036637    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036640    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036643    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036645    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036648    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036650    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036653    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036657    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036661    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036665    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036668    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036671    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036674    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036676    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036679    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:08:43.036714 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036681    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036684    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036687    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036689    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036692    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036694    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036697    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036699    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036702    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036704    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036707    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036710    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036712    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036715    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036718    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036721    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036724    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036727    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036730    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:08:43.037230 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036732    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036736    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036738    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036741    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036743    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036746    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036749    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036752    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036754    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036757    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036760    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036762    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036766    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036768    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036771    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036773    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036775    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036778    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036780    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036783    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:08:43.037719 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036786    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036788    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036791    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036794    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036796    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036799    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036801    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036806    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036810    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036813    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036816    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036819    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036822    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036825    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036828    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036830    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036833    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036836    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036839    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036841    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:08:43.038207 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036844    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036847    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036849    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036852    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036854    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036857    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036859    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036862    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036864    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036867    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036869    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036872    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036874    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036876    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036881    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036884    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036888    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:08:43.038705 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.036890    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:08:43.039156 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039143    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:08:43.039156 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039155    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039159    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039162    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039167    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039171    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039173    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039176    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039179    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039182    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039185    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039187    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039190    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039192    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039195    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039198    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039201    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039203    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039206    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039208    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:08:43.039210 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039211    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039213    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039216    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039219    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039222    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039225    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039227    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039230    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039232    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039235    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039237    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039240    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039243    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039245    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039248    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039250    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039253    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039256    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039258    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039261    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:08:43.039733 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039263    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039266    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039268    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039271    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039273    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039276    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039278    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039281    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039284    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039286    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039289    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039291    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039294    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039298    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039302    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039305    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039308    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039311    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039313    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039317    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:08:43.040331 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039320    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039323    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039326    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039328    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039331    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039333    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039336    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039339    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039341    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039344    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039346    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039349    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039351    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039354    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039357    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039359    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039361    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039364    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039366    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039369    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:08:43.040834 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039373    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039375    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039378    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039380    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039383    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039385    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039389    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039391    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039394    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039398    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039402    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039404    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039407    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039410    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039412    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039415    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039417    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039420    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039423    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:08:43.041319 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039801    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039806    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039809    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039811    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039814    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039817    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039819    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039822    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039824    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039827    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039830    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039833    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039835    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039838    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039841    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039844    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039846    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039849    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039852    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:08:43.041812 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039855    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039858    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039862    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039866    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039869    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039871    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039874    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039877    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039880    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039882    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039885    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039887    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039890    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039892    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039896    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039899    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039902    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039904    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039907    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:08:43.042264 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039909    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039912    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039914    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039917    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039919    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039922    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039927    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039930    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039933    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039936    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039939    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039942    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039945    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039947    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039950    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039953    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039956    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039958    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039961    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:08:43.042737 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039963    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039966    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039968    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039971    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039973    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039976    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039979    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039981    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039984    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039987    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039989    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039992    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039994    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.039998    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040000    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040002    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040005    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040007    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040010    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040013    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:08:43.043214 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040015    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040018    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040021    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040024    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040026    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040029    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040032    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040034    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040037    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040039    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040042    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040044    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040047    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040049    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040052    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040054    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040057    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040060    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040062    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040065    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040067    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:08:43.043709 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040069    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040136    2558 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040142    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040149    2558 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040156    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040163    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040168    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040174    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040180    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040183    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040186    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040190    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040193    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040196    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040200    2558 flags.go:64] FLAG: --cgroup-root=""
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040203    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040206    2558 flags.go:64] FLAG: --client-ca-file=""
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040209    2558 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040211    2558 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040215    2558 flags.go:64] FLAG: --cluster-domain=""
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040219    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040222    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040225    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040228    2558 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 20:08:43.044210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040232    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040235    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040238    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040241    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040244    2558 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040247    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040250    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040253    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040256    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040260    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040263    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040266    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040268    2558 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040271    2558 flags.go:64] FLAG: --enable-server="true"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040274    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040282    2558 flags.go:64] FLAG: --event-burst="100"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040286    2558 flags.go:64] FLAG: --event-qps="50"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040289    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040292    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040295    2558 flags.go:64] FLAG: --eviction-hard=""
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040298    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040301    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040304    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040308    2558 flags.go:64] FLAG: --eviction-soft=""
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040311    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 20:08:43.044841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040314    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040317    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040319    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040322    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040325    2558 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040328    2558 flags.go:64] FLAG: --feature-gates=""
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040332    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040335    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040338    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040341    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040344    2558 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040347    2558 flags.go:64] FLAG: --help="false"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040350    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040353    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040356    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040359    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040362    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040365    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040368    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040371    2558 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040374    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040377    2558 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040382    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040385    2558 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 20:08:43.045460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040388    2558 flags.go:64] FLAG: --kube-reserved=""
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040391    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040394    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040397    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040400    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040402    2558 flags.go:64] FLAG: --lock-file=""
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040406    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040409    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040412    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040430    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040434    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040438    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040441    2558 flags.go:64] FLAG: --logging-format="text"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040444    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040448    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040451    2558 flags.go:64] FLAG: --manifest-url=""
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040454    2558 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040462    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040464    2558 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040468    2558 flags.go:64] FLAG: --max-pods="110"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040471    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040474    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040477    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040480    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040483    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 20:08:43.046070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040486    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040489    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040496    2558 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040499    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040502    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040506    2558 flags.go:64] FLAG: --pod-cidr=""
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040509    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040515    2558 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040518    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040521    2558 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040524    2558 flags.go:64] FLAG: --port="10250"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040527    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040530    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-026e5d9d3e9fc0ee3"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040533    2558 flags.go:64] FLAG: --qos-reserved=""
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040536    2558 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040539    2558 flags.go:64] FLAG: --register-node="true"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040542    2558 flags.go:64] FLAG: --register-with-taints=""
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040546    2558 flags.go:64] FLAG: --registry-burst="10"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040548    2558 flags.go:64] FLAG: --registry-qps="5"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040551    2558 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040567    2558 flags.go:64] FLAG: --reserved-memory=""
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040571    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040574    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040577    2558 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040580    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 20:08:43.046679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040583    2558 flags.go:64] FLAG: --runonce="false"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040586    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040589    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040592    2558 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040595    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040600    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040603    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040606    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040609    2558 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040612    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040616    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040618    2558 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040622    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040625    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040628    2558 flags.go:64] FLAG: --system-cgroups=""
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040631    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040637    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040640    2558 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040642    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040647    2558 flags.go:64] FLAG: --tls-min-version=""
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040650    2558 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040653    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040656    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040659    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040662    2558 flags.go:64] FLAG: --v="2"
Jul 28 20:08:43.047465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040666    2558 flags.go:64] FLAG: --version="false"
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040670    2558 flags.go:64] FLAG: --vmodule=""
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040673    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.040676    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040796    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040800    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040803    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040806    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040810    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040812    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040815    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040818    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040823    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040825    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040828    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040831    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040833    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040836    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040839    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040843    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040847    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:08:43.048280 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040849    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040852    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040855    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040857    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040860    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040863    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040866    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040868    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040871    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040874    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040876    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040879    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040882    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040884    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040888    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040891    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040894    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040898    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040901    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:08:43.048862 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040904    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040907    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040910    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040912    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040916    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040919    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040921    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040924    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040927    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040929    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040932    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040936    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040939    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040942    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040944    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040947    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040949    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040952    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040954    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:08:43.049327 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040957    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040959    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040962    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040964    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040967    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040970    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040972    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040975    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040977    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040979    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040982    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040985    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040989    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040992    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040995    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.040998    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041001    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041006    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041009    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:08:43.049896 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041012    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041014    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041017    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041020    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041022    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041026    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041029    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041032    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041034    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041037    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041039    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041042    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041045    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041047    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041050    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041052    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041055    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041058    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041060    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041063    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:08:43.050361 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041066    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:08:43.050872 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041068    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:08:43.050872 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041071    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:08:43.050872 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041073    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:08:43.050872 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.041076    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:08:43.050872 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.041648    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 20:08:43.051440 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.051418    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 20:08:43.051440 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.051440    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 20:08:43.051513 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.051456    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 20:08:43.051513 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.051463    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051522    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051527    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051532    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051536    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051540    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051543    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051547    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051550    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051553    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051556    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051559    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051563    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051566    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051569    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051572    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:08:43.051568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051575    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051579    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051582    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051585    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051588    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051591    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051594    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051597    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051600    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051603    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051621    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051625    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051628    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051631    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051634    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051637    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051640    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051642    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051645    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:08:43.052012 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051648    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051651    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051654    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051657    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051659    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051662    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051665    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051668    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051671    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051674    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051677    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051680    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051682    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051686    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051689    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051692    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051695    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051698    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051700    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:08:43.052512 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051704    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051708    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051712    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051715    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051718    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051721    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051723    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051726    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051729    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051732    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051735    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051738    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051741    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051743    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051746    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051749    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051751    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051754    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051757    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:08:43.053077 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051760    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051763    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051766    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051769    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051772    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051775    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051778    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051793    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051797    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051800    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051802    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051805    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051808    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051811    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051814    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051817    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051819    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051822    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051825    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051827    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:08:43.053619 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051830    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051835    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051838    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051841    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051844    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051847    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051850    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.051856    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051977    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051983    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051986    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051989    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051992    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051996    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:08:43.054193 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.051999    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052002    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052005    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052008    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052011    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052013    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052023    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052027    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052030    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052032    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052035    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052038    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052041    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052044    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052046    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052049    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052052    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052055    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052058    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052061    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:08:43.054567 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052063    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052066    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052069    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052072    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052075    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052078    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052080    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052084    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052086    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052089    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052092    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052095    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052097    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052100    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052103    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052105    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052108    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052110    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052113    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052117    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:08:43.055185 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052120    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052123    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052125    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052128    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052131    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052134    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052137    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052139    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052142    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052144    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052147    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052150    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052152    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052155    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052158    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052160    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052164    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052166    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052169    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:08:43.055724 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052172    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052174    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052177    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052180    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052182    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052185    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052189    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052193    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052196    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052199    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052202    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052205    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052208    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052212    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052216    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052219    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052222    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052224    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052227    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:08:43.056232 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052230    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052233    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052236    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052239    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052243    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052246    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052249    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052253    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052256    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052259    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052262    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052266    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052269    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052272    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:08:43.056741 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:43.052275    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:08:43.057126 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.052280    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 20:08:43.057126 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.052386    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 20:08:43.057126 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.055302    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 20:08:43.057126 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.056726    2558 server.go:1019] "Starting client certificate rotation"
Jul 28 20:08:43.057126 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.056831    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 20:08:43.057126 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.056873    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 20:08:43.059276 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.059261    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 20:08:43.063128 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.063114    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 20:08:43.072581 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.072546    2558 log.go:25] "Validated CRI v1 runtime API"
Jul 28 20:08:43.076853 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.076835    2558 log.go:25] "Validated CRI v1 image API"
Jul 28 20:08:43.077851 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.077827    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 20:08:43.080455 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.080433    2558 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 af16d475-8d16-4ee4-a9ff-8666069de3c4:/dev/nvme0n1p4 f0265a66-8242-4d4e-a1c8-f10599183fb7:/dev/nvme0n1p3]
Jul 28 20:08:43.080519 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.080455    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 20:08:43.088031 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.087914    2558 manager.go:216] Machine: {Timestamp:2026-07-28 20:08:43.086447091 +0000 UTC m=+0.328540308 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105426 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29805b2ff3218b9b649a93c605b21d SystemUUID:ec29805b-2ff3-218b-9b64-9a93c605b21d BootID:380584e7-5bae-4d5e-83b0-475e2da848e0 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9a:18:3f:ff:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9a:18:3f:ff:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:7a:83:fb:dd:06 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 20:08:43.088031 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.088023    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 20:08:43.088160 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.088110    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 20:08:43.091223 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.091205    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 20:08:43.091584 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.091540    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 20:08:43.091735 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.091577    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-72.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 20:08:43.092380 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.092369    2558 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 20:08:43.092413 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.092383    2558 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 20:08:43.092413 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.092392    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 20:08:43.093364 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.093353    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 20:08:43.094115 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.094105    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 20:08:43.095905 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.095893    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 20:08:43.096021 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.096010    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 20:08:43.096066 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.096046    2558 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 20:08:43.096066 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.096056    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 20:08:43.096161 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.096075    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 20:08:43.096161 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.096085    2558 kubelet.go:398] "Adding apiserver pod source"
Jul 28 20:08:43.096161 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.096094    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 20:08:43.096457 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.096445    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 20:08:43.096497 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.096466    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 20:08:43.098398 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.098381    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 20:08:43.100422 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.100408    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 20:08:43.100474 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.100439    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 20:08:43.102159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.102148    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 20:08:43.102197 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.102165    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 20:08:43.102197 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.102171    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 20:08:43.102197 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.102177    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 20:08:43.102197 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.102188    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 20:08:43.102197 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.102197    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 20:08:43.102337 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.102208    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 20:08:43.102337 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.102213    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 20:08:43.102337 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.102221    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 20:08:43.102337 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.102228    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 20:08:43.102337 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.102242    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 20:08:43.102337 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.102249    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 20:08:43.103786 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.103772    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 20:08:43.103829 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.103788    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 20:08:43.107174 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.107157    2558 server.go:1268] "Started kubelet"
Jul 28 20:08:43.107294 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.107263    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 20:08:43.107425 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.107255    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 20:08:43.107515 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.107496    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 20:08:43.108145 ip-10-0-133-72 systemd[1]: Started Kubernetes Kubelet.
Jul 28 20:08:43.108786 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.108710    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-72.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 20:08:43.108786 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.108724    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-72.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 20:08:43.108890 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.108842    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 20:08:43.109406 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.109379    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 20:08:43.109969 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.109954    2558 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 20:08:43.111463 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.111434    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 20:08:43.115349 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.115330    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2qhbf"
Jul 28 20:08:43.116858 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.116839    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 20:08:43.116858 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.116850    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 20:08:43.117487 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.117420    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 20:08:43.117487 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.117454    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 20:08:43.117487 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.117440    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 20:08:43.117725 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.116971    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:43.117725 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.117592    2558 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 20:08:43.117725 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.117601    2558 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 20:08:43.117725 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.116608    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-72.ec2.internal.18c68cfd9833d63e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-72.ec2.internal,UID:ip-10-0-133-72.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-72.ec2.internal,},FirstTimestamp:2026-07-28 20:08:43.107128894 +0000 UTC m=+0.349222110,LastTimestamp:2026-07-28 20:08:43.107128894 +0000 UTC m=+0.349222110,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-72.ec2.internal,}"
Jul 28 20:08:43.118758 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.118740    2558 factory.go:55] Registering systemd factory
Jul 28 20:08:43.118831 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.118762    2558 factory.go:223] Registration of the systemd container factory successfully
Jul 28 20:08:43.118973 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.118959    2558 factory.go:153] Registering CRI-O factory
Jul 28 20:08:43.119059 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.118976    2558 factory.go:223] Registration of the crio container factory successfully
Jul 28 20:08:43.119059 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.119044    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 20:08:43.119157 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.119068    2558 factory.go:103] Registering Raw factory
Jul 28 20:08:43.119157 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.119087    2558 manager.go:1196] Started watching for new ooms in manager
Jul 28 20:08:43.119806 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.119784    2558 manager.go:349] Starting recovery of all containers
Jul 28 20:08:43.121057 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.121022    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-72.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 20:08:43.121212 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.121181    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 20:08:43.125268 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.125243    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2qhbf"
Jul 28 20:08:43.133498 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.133337    2558 manager.go:354] Recovery completed
Jul 28 20:08:43.135042 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.135024    2558 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 28 20:08:43.138210 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.138197    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:08:43.140020 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.140004    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:08:43.140112 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.140035    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:08:43.140112 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.140049    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:08:43.140618 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.140601    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 20:08:43.140618 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.140618    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 20:08:43.140715 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.140660    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 20:08:43.140715 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.140608    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:08:43.141468 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.141441    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:08:43.141545 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.141472    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:08:43.141545 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.141483    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:08:43.142191 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.142177    2558 policy_none.go:49] "None policy: Start"
Jul 28 20:08:43.142268 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.142199    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 20:08:43.142268 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.142213    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 20:08:43.142393 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.142331    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-72.ec2.internal.18c68cfd9a29b659  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-72.ec2.internal,UID:ip-10-0-133-72.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-72.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-72.ec2.internal,},FirstTimestamp:2026-07-28 20:08:43.140019801 +0000 UTC m=+0.382113021,LastTimestamp:2026-07-28 20:08:43.140019801 +0000 UTC m=+0.382113021,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-72.ec2.internal,}"
Jul 28 20:08:43.143376 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.143363    2558 policy_none.go:47] "Start"
Jul 28 20:08:43.180168 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.180153    2558 manager.go:337] "Starting Device Plugin manager"
Jul 28 20:08:43.180254 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.180192    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 20:08:43.180254 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.180202    2558 server.go:87] "Starting device plugin registration server"
Jul 28 20:08:43.180422 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.180409    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 20:08:43.180475 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.180422    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 20:08:43.180537 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.180521    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 20:08:43.180617 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.180604    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 20:08:43.180688 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.180617    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 20:08:43.181185 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.181159    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 20:08:43.181260 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.181222    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:43.282446 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.282398    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:08:43.283440 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.283323    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:08:43.283440 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.283349    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:08:43.283440 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.283360    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:08:43.283440 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.283390    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.291507 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.291491    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.291560 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.291514    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-72.ec2.internal\": node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:43.310158 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.310136    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:43.332225 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.332201    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 20:08:43.333720 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.333694    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 20:08:43.333805 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.333739    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 20:08:43.333805 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.333761    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 20:08:43.333805 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.333794    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 20:08:43.335849 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.335831    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:08:43.410814 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.410784    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:43.435052 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.435024    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-72.ec2.internal"]
Jul 28 20:08:43.435126 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.435112    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:08:43.437513 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.437498    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:08:43.437586 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.437527    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:08:43.437586 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.437538    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:08:43.438839 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.438826    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:08:43.438975 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.438963    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.439015 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.438992    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:08:43.439611 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.439591    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:08:43.439721 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.439627    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:08:43.439721 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.439645    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:08:43.439721 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.439645    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:08:43.439721 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.439673    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:08:43.439721 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.439684    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:08:43.440797 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.440782    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.440857 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.440809    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:08:43.441860 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.441841    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:08:43.441947 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.441871    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:08:43.441947 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.441881    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:08:43.459216 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.459197    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-72.ec2.internal\" not found" node="ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.464840 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.462940    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-72.ec2.internal\" not found" node="ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.511599 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.511573    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:43.519995 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.519975    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/048fa266a62c5ce6de745ad1e8c999a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal\" (UID: \"048fa266a62c5ce6de745ad1e8c999a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.520080 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.520005    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/048fa266a62c5ce6de745ad1e8c999a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal\" (UID: \"048fa266a62c5ce6de745ad1e8c999a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.520080 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.520024    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ada8b40b091a0bc1a7e4aff59957a1c4-config\") pod \"kube-apiserver-proxy-ip-10-0-133-72.ec2.internal\" (UID: \"ada8b40b091a0bc1a7e4aff59957a1c4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.611816 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.611737    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:43.620773 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.620755    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/048fa266a62c5ce6de745ad1e8c999a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal\" (UID: \"048fa266a62c5ce6de745ad1e8c999a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.620885 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.620780    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ada8b40b091a0bc1a7e4aff59957a1c4-config\") pod \"kube-apiserver-proxy-ip-10-0-133-72.ec2.internal\" (UID: \"ada8b40b091a0bc1a7e4aff59957a1c4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.620885 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.620799    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/048fa266a62c5ce6de745ad1e8c999a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal\" (UID: \"048fa266a62c5ce6de745ad1e8c999a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.620885 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.620838    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/048fa266a62c5ce6de745ad1e8c999a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal\" (UID: \"048fa266a62c5ce6de745ad1e8c999a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.620885 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.620865    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/048fa266a62c5ce6de745ad1e8c999a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal\" (UID: \"048fa266a62c5ce6de745ad1e8c999a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.620885 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.620882    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ada8b40b091a0bc1a7e4aff59957a1c4-config\") pod \"kube-apiserver-proxy-ip-10-0-133-72.ec2.internal\" (UID: \"ada8b40b091a0bc1a7e4aff59957a1c4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.716827 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.716792    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:43.763771 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.763740    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.766020 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:43.766003    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:43.818684 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.818657    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:43.919252 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:43.919188    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:44.019375 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:44.019351    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:44.055836 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:44.055808    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 20:08:44.056367 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:44.055955    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 20:08:44.120328 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:44.120298    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:44.130648 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:44.130622    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 20:03:44 +0000 UTC" deadline="2028-02-11 11:34:18.608172222 +0000 UTC"
Jul 28 20:08:44.130648 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:44.130647    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13503h25m34.477527957s"
Jul 28 20:08:44.167877 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:44.167853    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:08:44.220893 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:44.220824    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:44.321921 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:44.321900    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:44.414269 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:44.414234    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod048fa266a62c5ce6de745ad1e8c999a6.slice/crio-da4005b2967f7c2c22955754007692f81b6bc499a7b896a48046447e5f1d2546 WatchSource:0}: Error finding container da4005b2967f7c2c22955754007692f81b6bc499a7b896a48046447e5f1d2546: Status 404 returned error can't find the container with id da4005b2967f7c2c22955754007692f81b6bc499a7b896a48046447e5f1d2546
Jul 28 20:08:44.414952 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:44.414928    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada8b40b091a0bc1a7e4aff59957a1c4.slice/crio-a6aa4b95bf62b5b6ff891befdc21f27d593a88c13b7182aab65bea86e85d301d WatchSource:0}: Error finding container a6aa4b95bf62b5b6ff891befdc21f27d593a88c13b7182aab65bea86e85d301d: Status 404 returned error can't find the container with id a6aa4b95bf62b5b6ff891befdc21f27d593a88c13b7182aab65bea86e85d301d
Jul 28 20:08:44.420409 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:44.420391    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:08:44.422198 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:44.422178    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-72.ec2.internal\" not found"
Jul 28 20:08:44.448836 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:44.448817    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:08:44.522297 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:44.522268    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:44.532347 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:44.532326    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 20:08:44.533525 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:44.533512    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal"
Jul 28 20:08:44.543427 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:44.543411    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 20:08:44.606157 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:44.606137    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:08:45.098497 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.098464    2558 apiserver.go:52] "Watching apiserver"
Jul 28 20:08:45.107736 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.107499    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 20:08:45.108019 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.107988    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-z4zjn","openshift-multus/multus-additional-cni-plugins-xrgg9","openshift-multus/multus-b72f2","openshift-multus/network-metrics-daemon-pghh9","openshift-network-diagnostics/network-check-target-q9m9r","openshift-network-operator/iptables-alerter-dfchd","kube-system/global-pull-secret-syncer-zmkbq","kube-system/konnectivity-agent-w628c","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp","openshift-cluster-node-tuning-operator/tuned-zhhzv","openshift-dns/node-resolver-7hg4x","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-czcmx","kube-system/kube-apiserver-proxy-ip-10-0-133-72.ec2.internal"]
Jul 28 20:08:45.110166 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.110141    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:45.110298 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.110231    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:08:45.112571 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.111938    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.113814 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.113692    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.116322 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.115522    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 20:08:45.116322 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.115821    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 20:08:45.116322 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.115856    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:45.116322 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.115921    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:08:45.116322 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.116067    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 20:08:45.116322 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.116072    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 20:08:45.120519 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.118823    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:45.120519 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.118886    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:08:45.120519 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.118931    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 20:08:45.120519 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.119676    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dfchd"
Jul 28 20:08:45.123699 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.121180    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 20:08:45.123699 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.121477    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 20:08:45.123699 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.121631    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.123699 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.122972    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 20:08:45.123699 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.123008    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dhxpz\""
Jul 28 20:08:45.123699 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.123047    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 20:08:45.123699 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.123375    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 20:08:45.125046 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.123744    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 20:08:45.125046 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.123932    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j226p\""
Jul 28 20:08:45.125046 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.124037    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 20:08:45.125046 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.124367    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 20:08:45.125046 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.124434    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8fwkt\""
Jul 28 20:08:45.125564 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.125540    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 20:08:45.125851 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.125823    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z4zjn"
Jul 28 20:08:45.127463 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.127441    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 20:08:45.127866 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.127838    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 20:08:45.127866 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.127851    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 20:08:45.128800 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.128541    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rrvs7\""
Jul 28 20:08:45.128800 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.128567    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 20:08:45.128972 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.128959    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.129584    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-t2zdx\""
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.129820    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.129873    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-var-lib-openvswitch\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.129906    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85d75f9d-263e-430c-9493-429d83d28190-env-overrides\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.129933    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-multus-cni-dir\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.129961    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs\") pod \"network-metrics-daemon-pghh9\" (UID: \"232db419-1a78-4e78-a6fe-da6c6bc7e72f\") " pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.129990    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-run-ovn\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130056    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-systemd-units\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130094    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-run-netns\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130146    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85d75f9d-263e-430c-9493-429d83d28190-ovn-node-metrics-cert\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130176    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-cnibin\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130205    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-var-lib-cni-bin\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130230    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-multus-conf-dir\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130256    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-run-ovn-kubernetes\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130315    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-cni-bin\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.130351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130342    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b2678ad9-8742-49c3-98f7-7c22b69edb12-multus-daemon-config\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130370    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-etc-kubernetes\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130397    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr8c2\" (UniqueName: \"kubernetes.io/projected/232db419-1a78-4e78-a6fe-da6c6bc7e72f-kube-api-access-vr8c2\") pod \"network-metrics-daemon-pghh9\" (UID: \"232db419-1a78-4e78-a6fe-da6c6bc7e72f\") " pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130425    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgjvq\" (UniqueName: \"kubernetes.io/projected/fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5-kube-api-access-vgjvq\") pod \"iptables-alerter-dfchd\" (UID: \"fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5\") " pod="openshift-network-operator/iptables-alerter-dfchd"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130454    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1b683c6-ec1a-4f37-b354-3a366d2bd899-kubelet-config\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130492    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1b683c6-ec1a-4f37-b354-3a366d2bd899-dbus\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130538    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-slash\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130568    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-etc-openvswitch\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130597    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130726    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85d75f9d-263e-430c-9493-429d83d28190-ovnkube-config\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130755    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-os-release\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130782    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-var-lib-kubelet\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130810    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5-host-slash\") pod \"iptables-alerter-dfchd\" (UID: \"fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5\") " pod="openshift-network-operator/iptables-alerter-dfchd"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130843    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-run-systemd\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130893    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-run-multus-certs\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130926    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdwjr\" (UniqueName: \"kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr\") pod \"network-check-target-q9m9r\" (UID: \"8db6484b-7821-440f-b467-4aa1721dbd50\") " pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:45.131159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.130977    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-node-log\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131027    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-kubelet\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131071    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-run-openvswitch\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131098    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85d75f9d-263e-430c-9493-429d83d28190-ovnkube-script-lib\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131104    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131138    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2678ad9-8742-49c3-98f7-7c22b69edb12-cni-binary-copy\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131166    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-var-lib-cni-multus\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131191    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-hostroot\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131231    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-cni-netd\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131298    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kx2x\" (UniqueName: \"kubernetes.io/projected/b2678ad9-8742-49c3-98f7-7c22b69edb12-kube-api-access-5kx2x\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131335    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5-iptables-alerter-script\") pod \"iptables-alerter-dfchd\" (UID: \"fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5\") " pod="openshift-network-operator/iptables-alerter-dfchd"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131380    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131406    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-log-socket\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131439    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6xrx\" (UniqueName: \"kubernetes.io/projected/85d75f9d-263e-430c-9493-429d83d28190-kube-api-access-g6xrx\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131464    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-system-cni-dir\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131489    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-multus-socket-dir-parent\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131517    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-run-k8s-cni-cncf-io\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131560    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-run-netns\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.132027 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.131996    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 20:08:45.134405 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.132196    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7l6gm\""
Jul 28 20:08:45.134405 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.132407    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.134405 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.132628    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7hg4x"
Jul 28 20:08:45.134405 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.133327    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 20:08:45.134405 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.133516    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w628c"
Jul 28 20:08:45.135713 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.135247    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hpzgr\""
Jul 28 20:08:45.135713 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.135456    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 20:08:45.136239 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.136084    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cr9z2\""
Jul 28 20:08:45.136239 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.136135    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 20:08:45.136931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.136848    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 20:08:45.137710 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.137172    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 20:08:45.137710 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.137230    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 20:08:45.137710 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.137499    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5s7cb\""
Jul 28 20:08:45.137710 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.137565    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 20:08:45.138004 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.137898    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 20:08:45.167260 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.167225    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nl6ft"
Jul 28 20:08:45.174007 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.173859    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nl6ft"
Jul 28 20:08:45.219504 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.219481    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 20:08:45.232424 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.232386    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-run-netns\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.232531 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.232445    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-cnibin\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.232531 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.232485    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-var-lib-cni-bin\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.232685 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.232526    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-run-netns\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.232685 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.232544    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-cnibin\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.232685 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.232554    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-multus-conf-dir\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.233089 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233066    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-sys-fs\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.233156 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233118    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-run-ovn-kubernetes\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.233216 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233155    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr8c2\" (UniqueName: \"kubernetes.io/projected/232db419-1a78-4e78-a6fe-da6c6bc7e72f-kube-api-access-vr8c2\") pod \"network-metrics-daemon-pghh9\" (UID: \"232db419-1a78-4e78-a6fe-da6c6bc7e72f\") " pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:45.233216 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233181    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1b683c6-ec1a-4f37-b354-3a366d2bd899-kubelet-config\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:45.233216 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233204    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1b683c6-ec1a-4f37-b354-3a366d2bd899-dbus\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:45.233370 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233232    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-system-cni-dir\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.233370 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-sysctl-d\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.233370 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233281    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-host\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.233370 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233305    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85d75f9d-263e-430c-9493-429d83d28190-ovnkube-config\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.233370 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233304    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-multus-conf-dir\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.233370 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233330    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-var-lib-kubelet\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.233370 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233355    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-registration-dir\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.233370 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233357    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-var-lib-cni-bin\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233380    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1545e656-1004-4d5a-aee1-7bf87e8017b2-serviceca\") pod \"node-ca-z4zjn\" (UID: \"1545e656-1004-4d5a-aee1-7bf87e8017b2\") " pod="openshift-image-registry/node-ca-z4zjn"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233421    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-sysctl-conf\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233460    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdwjr\" (UniqueName: \"kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr\") pod \"network-check-target-q9m9r\" (UID: \"8db6484b-7821-440f-b467-4aa1721dbd50\") " pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233491    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-socket-dir\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233517    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-modprobe-d\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233547    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-run-openvswitch\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233575    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-var-lib-cni-multus\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233621    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kx2x\" (UniqueName: \"kubernetes.io/projected/b2678ad9-8742-49c3-98f7-7c22b69edb12-kube-api-access-5kx2x\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233648    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-cni-netd\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233689    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5-iptables-alerter-script\") pod \"iptables-alerter-dfchd\" (UID: \"fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5\") " pod="openshift-network-operator/iptables-alerter-dfchd"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-sys\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233744    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2bq8\" (UniqueName: \"kubernetes.io/projected/09f86b17-3df8-454e-a089-8b1c1056a282-kube-api-access-c2bq8\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233775    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/98a9c952-fdae-4bdd-9944-f68a30c41f83-hosts-file\") pod \"node-resolver-7hg4x\" (UID: \"98a9c952-fdae-4bdd-9944-f68a30c41f83\") " pod="openshift-dns/node-resolver-7hg4x"
Jul 28 20:08:45.233833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233804    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-log-socket\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233845    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6xrx\" (UniqueName: \"kubernetes.io/projected/85d75f9d-263e-430c-9493-429d83d28190-kube-api-access-g6xrx\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233880    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-system-cni-dir\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233915    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-multus-socket-dir-parent\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233944    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-run-netns\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233972    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-sysconfig\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.233998    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-run\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234033    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98a9c952-fdae-4bdd-9944-f68a30c41f83-tmp-dir\") pod \"node-resolver-7hg4x\" (UID: \"98a9c952-fdae-4bdd-9944-f68a30c41f83\") " pod="openshift-dns/node-resolver-7hg4x"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234045    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85d75f9d-263e-430c-9493-429d83d28190-ovnkube-config\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234061    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-multus-cni-dir\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234094    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs\") pod \"network-metrics-daemon-pghh9\" (UID: \"232db419-1a78-4e78-a6fe-da6c6bc7e72f\") " pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234137    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-var-lib-kubelet\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234170    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-device-dir\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234199    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-kubernetes\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.234209    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234224    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-var-lib-kubelet\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234252    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78srh\" (UniqueName: \"kubernetes.io/projected/98a9c952-fdae-4bdd-9944-f68a30c41f83-kube-api-access-78srh\") pod \"node-resolver-7hg4x\" (UID: \"98a9c952-fdae-4bdd-9944-f68a30c41f83\") " pod="openshift-dns/node-resolver-7hg4x"
Jul 28 20:08:45.234528 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234276    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4drt8\" (UniqueName: \"kubernetes.io/projected/8e9073fe-c018-418c-8a12-d53c52a1beee-kube-api-access-4drt8\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.235399 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.234306    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs podName:232db419-1a78-4e78-a6fe-da6c6bc7e72f nodeName:}" failed. No retries permitted until 2026-07-28 20:08:45.734283193 +0000 UTC m=+2.976376406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs") pod "network-metrics-daemon-pghh9" (UID: "232db419-1a78-4e78-a6fe-da6c6bc7e72f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:08:45.235399 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234360    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-run-ovn-kubernetes\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.235399 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234403    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-run-openvswitch\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.235399 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234460    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-var-lib-cni-multus\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.235399 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234625    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1b683c6-ec1a-4f37-b354-3a366d2bd899-kubelet-config\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:45.235399 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234739    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-cni-netd\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.235399 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.234784    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1b683c6-ec1a-4f37-b354-3a366d2bd899-dbus\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:45.235399 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235253    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-multus-socket-dir-parent\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.235399 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235300    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-log-socket\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.235399 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235322    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5-iptables-alerter-script\") pod \"iptables-alerter-dfchd\" (UID: \"fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5\") " pod="openshift-network-operator/iptables-alerter-dfchd"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235409    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-system-cni-dir\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235452    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-run-netns\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235509    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-multus-cni-dir\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235530    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-run-ovn\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235573    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-run-ovn\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235590    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85d75f9d-263e-430c-9493-429d83d28190-ovn-node-metrics-cert\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b2678ad9-8742-49c3-98f7-7c22b69edb12-multus-daemon-config\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235730    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-etc-kubernetes\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235761    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kgh8\" (UniqueName: \"kubernetes.io/projected/1545e656-1004-4d5a-aee1-7bf87e8017b2-kube-api-access-9kgh8\") pod \"node-ca-z4zjn\" (UID: \"1545e656-1004-4d5a-aee1-7bf87e8017b2\") " pod="openshift-image-registry/node-ca-z4zjn"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235790    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235820    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-cni-bin\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235848    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgjvq\" (UniqueName: \"kubernetes.io/projected/fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5-kube-api-access-vgjvq\") pod \"iptables-alerter-dfchd\" (UID: \"fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5\") " pod="openshift-network-operator/iptables-alerter-dfchd"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235875    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-cnibin\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.235931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235903    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66rgt\" (UniqueName: \"kubernetes.io/projected/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-kube-api-access-66rgt\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235943    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-slash\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.235983    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-etc-openvswitch\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236012    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236040    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-os-release\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236066    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5-host-slash\") pod \"iptables-alerter-dfchd\" (UID: \"fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5\") " pod="openshift-network-operator/iptables-alerter-dfchd"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236099    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1545e656-1004-4d5a-aee1-7bf87e8017b2-host\") pod \"node-ca-z4zjn\" (UID: \"1545e656-1004-4d5a-aee1-7bf87e8017b2\") " pod="openshift-image-registry/node-ca-z4zjn"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236132    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236159    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-lib-modules\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236188    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-run-systemd\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236201    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b2678ad9-8742-49c3-98f7-7c22b69edb12-multus-daemon-config\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236216    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-run-multus-certs\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236245    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-etc-selinux\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236276    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236302    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2692fbe8-7cff-4492-99fc-a18edc6b24fa-agent-certs\") pod \"konnectivity-agent-w628c\" (UID: \"2692fbe8-7cff-4492-99fc-a18edc6b24fa\") " pod="kube-system/konnectivity-agent-w628c"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236330    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2692fbe8-7cff-4492-99fc-a18edc6b24fa-konnectivity-ca\") pod \"konnectivity-agent-w628c\" (UID: \"2692fbe8-7cff-4492-99fc-a18edc6b24fa\") " pod="kube-system/konnectivity-agent-w628c"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236368    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09f86b17-3df8-454e-a089-8b1c1056a282-etc-tuned\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.236651 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236396    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-node-log\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.237465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236426    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-kubelet\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.237465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236455    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85d75f9d-263e-430c-9493-429d83d28190-ovnkube-script-lib\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.237465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236484    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2678ad9-8742-49c3-98f7-7c22b69edb12-cni-binary-copy\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.237465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236526    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-hostroot\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.237465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236559    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-os-release\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.237465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.236562    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 20:08:45.237465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.237102    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-etc-kubernetes\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.237465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.237153    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-cni-bin\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.237465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.237370    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-slash\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.237465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.237404    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-etc-openvswitch\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.237465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.237439    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.238071 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.237492    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-os-release\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.238071 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.237524    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5-host-slash\") pod \"iptables-alerter-dfchd\" (UID: \"fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5\") " pod="openshift-network-operator/iptables-alerter-dfchd"
Jul 28 20:08:45.238071 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.237586    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-run-systemd\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.238071 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.237625    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-run-multus-certs\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.238071 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.237742    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-host-kubelet\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.238071 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.237793    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-node-log\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.238347 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238210    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2678ad9-8742-49c3-98f7-7c22b69edb12-cni-binary-copy\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.238347 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.238347 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238302    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:45.238505 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238331    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-run-k8s-cni-cncf-io\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.238505 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238430    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.238505 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238454    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09f86b17-3df8-454e-a089-8b1c1056a282-tmp\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.238505 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238483    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-var-lib-openvswitch\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.238725 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238510    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85d75f9d-263e-430c-9493-429d83d28190-env-overrides\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.238725 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238541    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-systemd\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.238725 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238635    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-systemd-units\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.238725 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238669    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85d75f9d-263e-430c-9493-429d83d28190-ovnkube-script-lib\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.238930 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238727    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-systemd-units\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.238930 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.238746    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:08:45.238930 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238773    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-host-run-k8s-cni-cncf-io\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.238930 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238781    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85d75f9d-263e-430c-9493-429d83d28190-var-lib-openvswitch\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.238930 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.238809    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret podName:e1b683c6-ec1a-4f37-b354-3a366d2bd899 nodeName:}" failed. No retries permitted until 2026-07-28 20:08:45.738788527 +0000 UTC m=+2.980881755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret") pod "global-pull-secret-syncer-zmkbq" (UID: "e1b683c6-ec1a-4f37-b354-3a366d2bd899") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:08:45.238930 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.238868    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b2678ad9-8742-49c3-98f7-7c22b69edb12-hostroot\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.239243 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.239131    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85d75f9d-263e-430c-9493-429d83d28190-env-overrides\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.243047 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.243019    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85d75f9d-263e-430c-9493-429d83d28190-ovn-node-metrics-cert\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.244361 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.244341    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:08:45.244502 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.244487    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:08:45.244571 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.244531    2558 projected.go:196] Error preparing data for projected volume kube-api-access-qdwjr for pod openshift-network-diagnostics/network-check-target-q9m9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:08:45.244690 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.244617    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr podName:8db6484b-7821-440f-b467-4aa1721dbd50 nodeName:}" failed. No retries permitted until 2026-07-28 20:08:45.744576113 +0000 UTC m=+2.986669336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qdwjr" (UniqueName: "kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr") pod "network-check-target-q9m9r" (UID: "8db6484b-7821-440f-b467-4aa1721dbd50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:08:45.247764 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.247689    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr8c2\" (UniqueName: \"kubernetes.io/projected/232db419-1a78-4e78-a6fe-da6c6bc7e72f-kube-api-access-vr8c2\") pod \"network-metrics-daemon-pghh9\" (UID: \"232db419-1a78-4e78-a6fe-da6c6bc7e72f\") " pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:45.248153 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.248132    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kx2x\" (UniqueName: \"kubernetes.io/projected/b2678ad9-8742-49c3-98f7-7c22b69edb12-kube-api-access-5kx2x\") pod \"multus-b72f2\" (UID: \"b2678ad9-8742-49c3-98f7-7c22b69edb12\") " pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.249094 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.249063    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6xrx\" (UniqueName: \"kubernetes.io/projected/85d75f9d-263e-430c-9493-429d83d28190-kube-api-access-g6xrx\") pod \"ovnkube-node-czcmx\" (UID: \"85d75f9d-263e-430c-9493-429d83d28190\") " pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.250367 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.250343    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgjvq\" (UniqueName: \"kubernetes.io/projected/fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5-kube-api-access-vgjvq\") pod \"iptables-alerter-dfchd\" (UID: \"fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5\") " pod="openshift-network-operator/iptables-alerter-dfchd"
Jul 28 20:08:45.338953 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.338916    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-socket-dir\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.339144 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.338966    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-modprobe-d\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.339144 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339009    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-sys\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.339144 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339045    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2bq8\" (UniqueName: \"kubernetes.io/projected/09f86b17-3df8-454e-a089-8b1c1056a282-kube-api-access-c2bq8\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.339144 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339106    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/98a9c952-fdae-4bdd-9944-f68a30c41f83-hosts-file\") pod \"node-resolver-7hg4x\" (UID: \"98a9c952-fdae-4bdd-9944-f68a30c41f83\") " pod="openshift-dns/node-resolver-7hg4x"
Jul 28 20:08:45.339144 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339139    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-sysconfig\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.339416 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339166    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-run\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.339416 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339188    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98a9c952-fdae-4bdd-9944-f68a30c41f83-tmp-dir\") pod \"node-resolver-7hg4x\" (UID: \"98a9c952-fdae-4bdd-9944-f68a30c41f83\") " pod="openshift-dns/node-resolver-7hg4x"
Jul 28 20:08:45.339416 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339226    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-device-dir\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.339416 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339251    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-kubernetes\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.339416 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339278    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-var-lib-kubelet\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.339416 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339303    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78srh\" (UniqueName: \"kubernetes.io/projected/98a9c952-fdae-4bdd-9944-f68a30c41f83-kube-api-access-78srh\") pod \"node-resolver-7hg4x\" (UID: \"98a9c952-fdae-4bdd-9944-f68a30c41f83\") " pod="openshift-dns/node-resolver-7hg4x"
Jul 28 20:08:45.339416 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339325    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4drt8\" (UniqueName: \"kubernetes.io/projected/8e9073fe-c018-418c-8a12-d53c52a1beee-kube-api-access-4drt8\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.339416 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339349    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kgh8\" (UniqueName: \"kubernetes.io/projected/1545e656-1004-4d5a-aee1-7bf87e8017b2-kube-api-access-9kgh8\") pod \"node-ca-z4zjn\" (UID: \"1545e656-1004-4d5a-aee1-7bf87e8017b2\") " pod="openshift-image-registry/node-ca-z4zjn"
Jul 28 20:08:45.339416 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339369    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.339416 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339398    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-cnibin\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339422    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66rgt\" (UniqueName: \"kubernetes.io/projected/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-kube-api-access-66rgt\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339453    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1545e656-1004-4d5a-aee1-7bf87e8017b2-host\") pod \"node-ca-z4zjn\" (UID: \"1545e656-1004-4d5a-aee1-7bf87e8017b2\") " pod="openshift-image-registry/node-ca-z4zjn"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339483    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339508    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-lib-modules\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339536    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-etc-selinux\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339566    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339553    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-device-dir\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339592    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2692fbe8-7cff-4492-99fc-a18edc6b24fa-agent-certs\") pod \"konnectivity-agent-w628c\" (UID: \"2692fbe8-7cff-4492-99fc-a18edc6b24fa\") " pod="kube-system/konnectivity-agent-w628c"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339618    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2692fbe8-7cff-4492-99fc-a18edc6b24fa-konnectivity-ca\") pod \"konnectivity-agent-w628c\" (UID: \"2692fbe8-7cff-4492-99fc-a18edc6b24fa\") " pod="kube-system/konnectivity-agent-w628c"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339663    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09f86b17-3df8-454e-a089-8b1c1056a282-etc-tuned\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339696    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-os-release\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339721    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339731    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-modprobe-d\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339760    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339784    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09f86b17-3df8-454e-a089-8b1c1056a282-tmp\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339788    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-sys\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.339938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339812    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-systemd\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339852    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-sys-fs\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339885    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-system-cni-dir\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339911    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-sysctl-d\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339936    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-host\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339963    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-registration-dir\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.339990    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1545e656-1004-4d5a-aee1-7bf87e8017b2-serviceca\") pod \"node-ca-z4zjn\" (UID: \"1545e656-1004-4d5a-aee1-7bf87e8017b2\") " pod="openshift-image-registry/node-ca-z4zjn"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.340018    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-sysctl-conf\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.340145    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-run\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.340149    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/98a9c952-fdae-4bdd-9944-f68a30c41f83-hosts-file\") pod \"node-resolver-7hg4x\" (UID: \"98a9c952-fdae-4bdd-9944-f68a30c41f83\") " pod="openshift-dns/node-resolver-7hg4x"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.340205    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-sysctl-conf\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.340225    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-sysconfig\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.340267    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-kubernetes\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.340345    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-socket-dir\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.340406    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-var-lib-kubelet\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.340441    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98a9c952-fdae-4bdd-9944-f68a30c41f83-tmp-dir\") pod \"node-resolver-7hg4x\" (UID: \"98a9c952-fdae-4bdd-9944-f68a30c41f83\") " pod="openshift-dns/node-resolver-7hg4x"
Jul 28 20:08:45.340822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.340524    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-systemd\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.341722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.340846    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2692fbe8-7cff-4492-99fc-a18edc6b24fa-konnectivity-ca\") pod \"konnectivity-agent-w628c\" (UID: \"2692fbe8-7cff-4492-99fc-a18edc6b24fa\") " pod="kube-system/konnectivity-agent-w628c"
Jul 28 20:08:45.341722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.340929    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-sys-fs\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.341722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.340968    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-system-cni-dir\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.341722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.341079    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-etc-sysctl-d\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.341722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.341130    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-registration-dir\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.341722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.341098    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal" event={"ID":"048fa266a62c5ce6de745ad1e8c999a6","Type":"ContainerStarted","Data":"da4005b2967f7c2c22955754007692f81b6bc499a7b896a48046447e5f1d2546"}
Jul 28 20:08:45.341722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.341295    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1545e656-1004-4d5a-aee1-7bf87e8017b2-host\") pod \"node-ca-z4zjn\" (UID: \"1545e656-1004-4d5a-aee1-7bf87e8017b2\") " pod="openshift-image-registry/node-ca-z4zjn"
Jul 28 20:08:45.341722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.341535    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1545e656-1004-4d5a-aee1-7bf87e8017b2-serviceca\") pod \"node-ca-z4zjn\" (UID: \"1545e656-1004-4d5a-aee1-7bf87e8017b2\") " pod="openshift-image-registry/node-ca-z4zjn"
Jul 28 20:08:45.342129 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.341744    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.342129 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.341753    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.342129 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.341767    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.342129 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.341829    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-host\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.342129 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.341839    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-os-release\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.342129 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.341940    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09f86b17-3df8-454e-a089-8b1c1056a282-lib-modules\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.342129 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.341947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8e9073fe-c018-418c-8a12-d53c52a1beee-etc-selinux\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.342129 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.342043    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-cnibin\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.342129 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.342077    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.343518 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.343468    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09f86b17-3df8-454e-a089-8b1c1056a282-etc-tuned\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.343965 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.343939    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.344262 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.344242    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-72.ec2.internal" event={"ID":"ada8b40b091a0bc1a7e4aff59957a1c4","Type":"ContainerStarted","Data":"a6aa4b95bf62b5b6ff891befdc21f27d593a88c13b7182aab65bea86e85d301d"}
Jul 28 20:08:45.345841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.345797    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2692fbe8-7cff-4492-99fc-a18edc6b24fa-agent-certs\") pod \"konnectivity-agent-w628c\" (UID: \"2692fbe8-7cff-4492-99fc-a18edc6b24fa\") " pod="kube-system/konnectivity-agent-w628c"
Jul 28 20:08:45.346143 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.346121    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09f86b17-3df8-454e-a089-8b1c1056a282-tmp\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.348991 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.348930    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2bq8\" (UniqueName: \"kubernetes.io/projected/09f86b17-3df8-454e-a089-8b1c1056a282-kube-api-access-c2bq8\") pod \"tuned-zhhzv\" (UID: \"09f86b17-3df8-454e-a089-8b1c1056a282\") " pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.349857 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.349803    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78srh\" (UniqueName: \"kubernetes.io/projected/98a9c952-fdae-4bdd-9944-f68a30c41f83-kube-api-access-78srh\") pod \"node-resolver-7hg4x\" (UID: \"98a9c952-fdae-4bdd-9944-f68a30c41f83\") " pod="openshift-dns/node-resolver-7hg4x"
Jul 28 20:08:45.350039 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.350016    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66rgt\" (UniqueName: \"kubernetes.io/projected/173e52b7-2fc0-4c1a-8036-96d743ff8dd1-kube-api-access-66rgt\") pod \"multus-additional-cni-plugins-xrgg9\" (UID: \"173e52b7-2fc0-4c1a-8036-96d743ff8dd1\") " pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.351403 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.351378    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4drt8\" (UniqueName: \"kubernetes.io/projected/8e9073fe-c018-418c-8a12-d53c52a1beee-kube-api-access-4drt8\") pod \"aws-ebs-csi-driver-node-thrlp\" (UID: \"8e9073fe-c018-418c-8a12-d53c52a1beee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.352413 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.352366    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kgh8\" (UniqueName: \"kubernetes.io/projected/1545e656-1004-4d5a-aee1-7bf87e8017b2-kube-api-access-9kgh8\") pod \"node-ca-z4zjn\" (UID: \"1545e656-1004-4d5a-aee1-7bf87e8017b2\") " pod="openshift-image-registry/node-ca-z4zjn"
Jul 28 20:08:45.437285 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.437249    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:08:45.448572 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.448532    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b72f2"
Jul 28 20:08:45.459449 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.459423    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dfchd"
Jul 28 20:08:45.468440 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.468416    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp"
Jul 28 20:08:45.476375 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.476352    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z4zjn"
Jul 28 20:08:45.487421 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.487396    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xrgg9"
Jul 28 20:08:45.493356 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.493334    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zhhzv"
Jul 28 20:08:45.504011 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.503993    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7hg4x"
Jul 28 20:08:45.510060 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.510043    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w628c"
Jul 28 20:08:45.649816 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.649726    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:08:45.743410 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.743366    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs\") pod \"network-metrics-daemon-pghh9\" (UID: \"232db419-1a78-4e78-a6fe-da6c6bc7e72f\") " pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:45.743611 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.743425    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:45.743611 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.743532    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:08:45.743611 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.743579    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:08:45.743611 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.743588    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret podName:e1b683c6-ec1a-4f37-b354-3a366d2bd899 nodeName:}" failed. No retries permitted until 2026-07-28 20:08:46.743573921 +0000 UTC m=+3.985667124 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret") pod "global-pull-secret-syncer-zmkbq" (UID: "e1b683c6-ec1a-4f37-b354-3a366d2bd899") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:08:45.743827 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.743651    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs podName:232db419-1a78-4e78-a6fe-da6c6bc7e72f nodeName:}" failed. No retries permitted until 2026-07-28 20:08:46.743630115 +0000 UTC m=+3.985723339 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs") pod "network-metrics-daemon-pghh9" (UID: "232db419-1a78-4e78-a6fe-da6c6bc7e72f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:08:45.844374 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:45.844338    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdwjr\" (UniqueName: \"kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr\") pod \"network-check-target-q9m9r\" (UID: \"8db6484b-7821-440f-b467-4aa1721dbd50\") " pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:45.844536 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.844480    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:08:45.844536 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.844498    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:08:45.844536 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.844507    2558 projected.go:196] Error preparing data for projected volume kube-api-access-qdwjr for pod openshift-network-diagnostics/network-check-target-q9m9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:08:45.844679 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:45.844562    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr podName:8db6484b-7821-440f-b467-4aa1721dbd50 nodeName:}" failed. No retries permitted until 2026-07-28 20:08:46.84454498 +0000 UTC m=+4.086638182 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qdwjr" (UniqueName: "kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr") pod "network-check-target-q9m9r" (UID: "8db6484b-7821-440f-b467-4aa1721dbd50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:08:46.176888 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:46.176827    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 20:03:45 +0000 UTC" deadline="2028-01-17 12:22:33.120495826 +0000 UTC"
Jul 28 20:08:46.176888 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:46.176878    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12904h13m46.943622422s"
Jul 28 20:08:46.335049 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:46.335015    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:46.335207 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:46.335015    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:46.335207 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:46.335152    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:08:46.335207 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:46.335200    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:08:46.488922 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:46.488892    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a9c952_fdae_4bdd_9944_f68a30c41f83.slice/crio-280d99b6bf7d9ed5544b08c587c5bbcdc40b4ff3d08120e45660c52225259f7a WatchSource:0}: Error finding container 280d99b6bf7d9ed5544b08c587c5bbcdc40b4ff3d08120e45660c52225259f7a: Status 404 returned error can't find the container with id 280d99b6bf7d9ed5544b08c587c5bbcdc40b4ff3d08120e45660c52225259f7a
Jul 28 20:08:46.490033 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:46.489981    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod173e52b7_2fc0_4c1a_8036_96d743ff8dd1.slice/crio-b66fb442377f23f5fa08a153b8a2391cc9fb5b32af229cfb9d6e25a04d22249e WatchSource:0}: Error finding container b66fb442377f23f5fa08a153b8a2391cc9fb5b32af229cfb9d6e25a04d22249e: Status 404 returned error can't find the container with id b66fb442377f23f5fa08a153b8a2391cc9fb5b32af229cfb9d6e25a04d22249e
Jul 28 20:08:46.491043 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:46.491018    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d75f9d_263e_430c_9493_429d83d28190.slice/crio-94d31610f646e10a3cd71722b75f3ba90669f8828ca13c790e764cba7e365e8e WatchSource:0}: Error finding container 94d31610f646e10a3cd71722b75f3ba90669f8828ca13c790e764cba7e365e8e: Status 404 returned error can't find the container with id 94d31610f646e10a3cd71722b75f3ba90669f8828ca13c790e764cba7e365e8e
Jul 28 20:08:46.492843 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:46.492802    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2692fbe8_7cff_4492_99fc_a18edc6b24fa.slice/crio-0416e6b070440f9b4f52f08bb04a65d600995f7c2ebebb97ce0e1d679825085a WatchSource:0}: Error finding container 0416e6b070440f9b4f52f08bb04a65d600995f7c2ebebb97ce0e1d679825085a: Status 404 returned error can't find the container with id 0416e6b070440f9b4f52f08bb04a65d600995f7c2ebebb97ce0e1d679825085a
Jul 28 20:08:46.494274 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:46.493703    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1545e656_1004_4d5a_aee1_7bf87e8017b2.slice/crio-6e855995a1af509376d8a9778ddf95849f1a4686be66ab1a43a6dbf5d4771375 WatchSource:0}: Error finding container 6e855995a1af509376d8a9778ddf95849f1a4686be66ab1a43a6dbf5d4771375: Status 404 returned error can't find the container with id 6e855995a1af509376d8a9778ddf95849f1a4686be66ab1a43a6dbf5d4771375
Jul 28 20:08:46.497212 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:46.497190    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f86b17_3df8_454e_a089_8b1c1056a282.slice/crio-1091739b19dec1d97ccb6d50eb75a01898c83fe88ba608417d79f88fb4e0d564 WatchSource:0}: Error finding container 1091739b19dec1d97ccb6d50eb75a01898c83fe88ba608417d79f88fb4e0d564: Status 404 returned error can't find the container with id 1091739b19dec1d97ccb6d50eb75a01898c83fe88ba608417d79f88fb4e0d564
Jul 28 20:08:46.498578 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:46.498375    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2678ad9_8742_49c3_98f7_7c22b69edb12.slice/crio-599f174a04db8407f3b2a29d2ff32de6532b3a9e9aed5e6d4e4c2d5c91710bd8 WatchSource:0}: Error finding container 599f174a04db8407f3b2a29d2ff32de6532b3a9e9aed5e6d4e4c2d5c91710bd8: Status 404 returned error can't find the container with id 599f174a04db8407f3b2a29d2ff32de6532b3a9e9aed5e6d4e4c2d5c91710bd8
Jul 28 20:08:46.499998 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:46.499309    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e9073fe_c018_418c_8a12_d53c52a1beee.slice/crio-0e9348f3ed557854342a892318276a8a2cc835a862ab3f159ed7f26309d5e8d9 WatchSource:0}: Error finding container 0e9348f3ed557854342a892318276a8a2cc835a862ab3f159ed7f26309d5e8d9: Status 404 returned error can't find the container with id 0e9348f3ed557854342a892318276a8a2cc835a862ab3f159ed7f26309d5e8d9
Jul 28 20:08:46.501029 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:08:46.500955    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe05f34c_5651_4eb1_ab1a_cc871b2bfcc5.slice/crio-afc7d1b8a14de79b994bded58980cc9cad5f3ad27d9d87765c5e68f602b9b1ac WatchSource:0}: Error finding container afc7d1b8a14de79b994bded58980cc9cad5f3ad27d9d87765c5e68f602b9b1ac: Status 404 returned error can't find the container with id afc7d1b8a14de79b994bded58980cc9cad5f3ad27d9d87765c5e68f602b9b1ac
Jul 28 20:08:46.751326 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:46.751160    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs\") pod \"network-metrics-daemon-pghh9\" (UID: \"232db419-1a78-4e78-a6fe-da6c6bc7e72f\") " pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:46.751326 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:46.751319    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:08:46.751491 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:46.751357    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:46.751491 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:46.751390    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs podName:232db419-1a78-4e78-a6fe-da6c6bc7e72f nodeName:}" failed. No retries permitted until 2026-07-28 20:08:48.751364393 +0000 UTC m=+5.993457611 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs") pod "network-metrics-daemon-pghh9" (UID: "232db419-1a78-4e78-a6fe-da6c6bc7e72f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:08:46.751491 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:46.751450    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:08:46.751491 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:46.751495    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret podName:e1b683c6-ec1a-4f37-b354-3a366d2bd899 nodeName:}" failed. No retries permitted until 2026-07-28 20:08:48.751483331 +0000 UTC m=+5.993576535 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret") pod "global-pull-secret-syncer-zmkbq" (UID: "e1b683c6-ec1a-4f37-b354-3a366d2bd899") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:08:46.852492 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:46.852462    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdwjr\" (UniqueName: \"kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr\") pod \"network-check-target-q9m9r\" (UID: \"8db6484b-7821-440f-b467-4aa1721dbd50\") " pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:46.852709 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:46.852686    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:08:46.852801 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:46.852717    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:08:46.852801 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:46.852733    2558 projected.go:196] Error preparing data for projected volume kube-api-access-qdwjr for pod openshift-network-diagnostics/network-check-target-q9m9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:08:46.852922 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:46.852810    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr podName:8db6484b-7821-440f-b467-4aa1721dbd50 nodeName:}" failed. No retries permitted until 2026-07-28 20:08:48.852790342 +0000 UTC m=+6.094883563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qdwjr" (UniqueName: "kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr") pod "network-check-target-q9m9r" (UID: "8db6484b-7821-440f-b467-4aa1721dbd50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:08:47.177430 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.177384    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 20:03:45 +0000 UTC" deadline="2028-05-11 20:40:27.39099389 +0000 UTC"
Jul 28 20:08:47.177430 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.177428    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15672h31m40.21357087s"
Jul 28 20:08:47.335633 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.335072    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:47.335633 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:47.335221    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:08:47.354831 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.354795    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7hg4x" event={"ID":"98a9c952-fdae-4bdd-9944-f68a30c41f83","Type":"ContainerStarted","Data":"280d99b6bf7d9ed5544b08c587c5bbcdc40b4ff3d08120e45660c52225259f7a"}
Jul 28 20:08:47.368279 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.368247    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-72.ec2.internal" event={"ID":"ada8b40b091a0bc1a7e4aff59957a1c4","Type":"ContainerStarted","Data":"263238e26bc6fe4f43035947bd16fcdb082e028115dde6a3ab647fea76d2ca3b"}
Jul 28 20:08:47.376838 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.376769    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dfchd" event={"ID":"fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5","Type":"ContainerStarted","Data":"afc7d1b8a14de79b994bded58980cc9cad5f3ad27d9d87765c5e68f602b9b1ac"}
Jul 28 20:08:47.382498 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.382442    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w628c" event={"ID":"2692fbe8-7cff-4492-99fc-a18edc6b24fa","Type":"ContainerStarted","Data":"0416e6b070440f9b4f52f08bb04a65d600995f7c2ebebb97ce0e1d679825085a"}
Jul 28 20:08:47.404831 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.404803    2558 generic.go:356] "Generic (PLEG): container finished" podID="048fa266a62c5ce6de745ad1e8c999a6" containerID="77161370444f6575ea3d35181e3d41cb1a30819b41c546ec8746f2aa6333362c" exitCode=0
Jul 28 20:08:47.404928 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.404883    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal" event={"ID":"048fa266a62c5ce6de745ad1e8c999a6","Type":"ContainerDied","Data":"77161370444f6575ea3d35181e3d41cb1a30819b41c546ec8746f2aa6333362c"}
Jul 28 20:08:47.409305 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.409277    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b72f2" event={"ID":"b2678ad9-8742-49c3-98f7-7c22b69edb12","Type":"ContainerStarted","Data":"599f174a04db8407f3b2a29d2ff32de6532b3a9e9aed5e6d4e4c2d5c91710bd8"}
Jul 28 20:08:47.426661 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.426612    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp" event={"ID":"8e9073fe-c018-418c-8a12-d53c52a1beee","Type":"ContainerStarted","Data":"0e9348f3ed557854342a892318276a8a2cc835a862ab3f159ed7f26309d5e8d9"}
Jul 28 20:08:47.432496 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.432156    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-72.ec2.internal" podStartSLOduration=2.432137545 podStartE2EDuration="2.432137545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:08:47.388106121 +0000 UTC m=+4.630199350" watchObservedRunningTime="2026-07-28 20:08:47.432137545 +0000 UTC m=+4.674230773"
Jul 28 20:08:47.439114 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.439085    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zhhzv" event={"ID":"09f86b17-3df8-454e-a089-8b1c1056a282","Type":"ContainerStarted","Data":"1091739b19dec1d97ccb6d50eb75a01898c83fe88ba608417d79f88fb4e0d564"}
Jul 28 20:08:47.450448 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.450421    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z4zjn" event={"ID":"1545e656-1004-4d5a-aee1-7bf87e8017b2","Type":"ContainerStarted","Data":"6e855995a1af509376d8a9778ddf95849f1a4686be66ab1a43a6dbf5d4771375"}
Jul 28 20:08:47.471839 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.471739    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx" event={"ID":"85d75f9d-263e-430c-9493-429d83d28190","Type":"ContainerStarted","Data":"94d31610f646e10a3cd71722b75f3ba90669f8828ca13c790e764cba7e365e8e"}
Jul 28 20:08:47.494085 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:47.493951    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrgg9" event={"ID":"173e52b7-2fc0-4c1a-8036-96d743ff8dd1","Type":"ContainerStarted","Data":"b66fb442377f23f5fa08a153b8a2391cc9fb5b32af229cfb9d6e25a04d22249e"}
Jul 28 20:08:48.334776 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:48.334737    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:48.335292 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:48.334903    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:08:48.335292 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:48.334739    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:48.335409 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:48.335357    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:08:48.527182 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:48.526714    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal" event={"ID":"048fa266a62c5ce6de745ad1e8c999a6","Type":"ContainerStarted","Data":"b562325403b9ee6a155cc6341767786aebd0fe27201db50f1b47223c898d3ea9"}
Jul 28 20:08:48.544412 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:48.544356    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-72.ec2.internal" podStartSLOduration=3.54433752 podStartE2EDuration="3.54433752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:08:48.543695538 +0000 UTC m=+5.785788768" watchObservedRunningTime="2026-07-28 20:08:48.54433752 +0000 UTC m=+5.786430765"
Jul 28 20:08:48.769332 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:48.769298    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:48.769512 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:48.769380    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs\") pod \"network-metrics-daemon-pghh9\" (UID: \"232db419-1a78-4e78-a6fe-da6c6bc7e72f\") " pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:48.769594 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:48.769511    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:08:48.769594 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:48.769571    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs podName:232db419-1a78-4e78-a6fe-da6c6bc7e72f nodeName:}" failed. No retries permitted until 2026-07-28 20:08:52.769552087 +0000 UTC m=+10.011645293 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs") pod "network-metrics-daemon-pghh9" (UID: "232db419-1a78-4e78-a6fe-da6c6bc7e72f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:08:48.769992 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:48.769972    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:08:48.770087 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:48.770031    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret podName:e1b683c6-ec1a-4f37-b354-3a366d2bd899 nodeName:}" failed. No retries permitted until 2026-07-28 20:08:52.770014953 +0000 UTC m=+10.012108163 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret") pod "global-pull-secret-syncer-zmkbq" (UID: "e1b683c6-ec1a-4f37-b354-3a366d2bd899") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:08:48.870447 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:48.870378    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdwjr\" (UniqueName: \"kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr\") pod \"network-check-target-q9m9r\" (UID: \"8db6484b-7821-440f-b467-4aa1721dbd50\") " pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:48.870736 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:48.870551    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:08:48.870736 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:48.870575    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:08:48.870736 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:48.870590    2558 projected.go:196] Error preparing data for projected volume kube-api-access-qdwjr for pod openshift-network-diagnostics/network-check-target-q9m9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:08:48.870736 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:48.870675    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr podName:8db6484b-7821-440f-b467-4aa1721dbd50 nodeName:}" failed. No retries permitted until 2026-07-28 20:08:52.870649487 +0000 UTC m=+10.112742714 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qdwjr" (UniqueName: "kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr") pod "network-check-target-q9m9r" (UID: "8db6484b-7821-440f-b467-4aa1721dbd50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:08:49.335881 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:49.335846    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:49.336394 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:49.335986    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:08:50.334392 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:50.334359    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:50.334593 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:50.334409    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:50.334593 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:50.334505    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:08:50.334738 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:50.334657    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:08:51.334948 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:51.334907    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:51.335388 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:51.335054    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:08:52.334911 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:52.334868    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:52.335096 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:52.334868    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:52.335468 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:52.335137    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:08:52.335468 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:52.335311    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:08:52.805890 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:52.805822    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:52.806084 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:52.805979    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:08:52.806084 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:52.805996    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs\") pod \"network-metrics-daemon-pghh9\" (UID: \"232db419-1a78-4e78-a6fe-da6c6bc7e72f\") " pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:52.806084 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:52.806056    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret podName:e1b683c6-ec1a-4f37-b354-3a366d2bd899 nodeName:}" failed. No retries permitted until 2026-07-28 20:09:00.80603318 +0000 UTC m=+18.048126395 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret") pod "global-pull-secret-syncer-zmkbq" (UID: "e1b683c6-ec1a-4f37-b354-3a366d2bd899") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:08:52.806275 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:52.806125    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:08:52.806275 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:52.806213    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs podName:232db419-1a78-4e78-a6fe-da6c6bc7e72f nodeName:}" failed. No retries permitted until 2026-07-28 20:09:00.80616493 +0000 UTC m=+18.048258157 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs") pod "network-metrics-daemon-pghh9" (UID: "232db419-1a78-4e78-a6fe-da6c6bc7e72f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:08:52.907278 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:52.907243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdwjr\" (UniqueName: \"kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr\") pod \"network-check-target-q9m9r\" (UID: \"8db6484b-7821-440f-b467-4aa1721dbd50\") " pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:52.907475 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:52.907406    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:08:52.907475 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:52.907426    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:08:52.907475 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:52.907438    2558 projected.go:196] Error preparing data for projected volume kube-api-access-qdwjr for pod openshift-network-diagnostics/network-check-target-q9m9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:08:52.907699 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:52.907504    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr podName:8db6484b-7821-440f-b467-4aa1721dbd50 nodeName:}" failed. No retries permitted until 2026-07-28 20:09:00.907485285 +0000 UTC m=+18.149578514 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qdwjr" (UniqueName: "kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr") pod "network-check-target-q9m9r" (UID: "8db6484b-7821-440f-b467-4aa1721dbd50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:08:53.335407 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:53.335375    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:53.335987 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:53.335494    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:08:54.334046 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:54.334002    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:54.334241 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:54.334162    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:08:54.334319 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:54.334255    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:54.334356 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:54.334332    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:08:55.336588 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:55.336529    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:55.337092 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:55.336692    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:08:56.334389 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:56.334355    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:56.334531 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:56.334506    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:08:56.334626 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:56.334596    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:56.334720 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:56.334697    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:08:57.334365 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:57.334310    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:57.334760 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:57.334473    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:08:58.334263 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:58.334234    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:08:58.334431 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:58.334234    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:08:58.334431 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:58.334338    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:08:58.334790 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:58.334459    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:08:59.334291 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:08:59.334256    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:08:59.334473 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:08:59.334362    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:00.334162 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:00.334127    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:00.334324 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:00.334174    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:00.334324 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:00.334251    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:00.334435 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:00.334325    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:00.870155 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:00.870119    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs\") pod \"network-metrics-daemon-pghh9\" (UID: \"232db419-1a78-4e78-a6fe-da6c6bc7e72f\") " pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:00.870658 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:00.870165    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:00.870658 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:00.870256    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:09:00.870658 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:00.870265    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:09:00.870658 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:00.870326    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs podName:232db419-1a78-4e78-a6fe-da6c6bc7e72f nodeName:}" failed. No retries permitted until 2026-07-28 20:09:16.870306324 +0000 UTC m=+34.112399541 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs") pod "network-metrics-daemon-pghh9" (UID: "232db419-1a78-4e78-a6fe-da6c6bc7e72f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:09:00.870658 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:00.870343    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret podName:e1b683c6-ec1a-4f37-b354-3a366d2bd899 nodeName:}" failed. No retries permitted until 2026-07-28 20:09:16.870336334 +0000 UTC m=+34.112429535 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret") pod "global-pull-secret-syncer-zmkbq" (UID: "e1b683c6-ec1a-4f37-b354-3a366d2bd899") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:09:00.971054 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:00.971022    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdwjr\" (UniqueName: \"kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr\") pod \"network-check-target-q9m9r\" (UID: \"8db6484b-7821-440f-b467-4aa1721dbd50\") " pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:00.971234 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:00.971173    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:09:00.971234 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:00.971194    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:09:00.971234 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:00.971208    2558 projected.go:196] Error preparing data for projected volume kube-api-access-qdwjr for pod openshift-network-diagnostics/network-check-target-q9m9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:09:00.971395 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:00.971263    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr podName:8db6484b-7821-440f-b467-4aa1721dbd50 nodeName:}" failed. No retries permitted until 2026-07-28 20:09:16.971245772 +0000 UTC m=+34.213338973 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qdwjr" (UniqueName: "kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr") pod "network-check-target-q9m9r" (UID: "8db6484b-7821-440f-b467-4aa1721dbd50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:09:01.334388 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:01.334362    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:01.334524 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:01.334465    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:02.334451 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:02.334407    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:02.334451 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:02.334455    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:02.335003 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:02.334582    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:02.335003 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:02.334695    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:03.335573 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:03.335519    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:03.335984 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:03.335665    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:04.334408 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:04.334367    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:04.334615 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:04.334415    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:04.334615 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:04.334523    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:04.334741 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:04.334679    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:05.337433 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.334973    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:05.337433 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:05.335333    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:05.563364 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.563319    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zhhzv" event={"ID":"09f86b17-3df8-454e-a089-8b1c1056a282","Type":"ContainerStarted","Data":"51fef9d56912067864e5fa18f227770404ed74133e1d3aee18bf6d772370be88"}
Jul 28 20:09:05.565168 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.565140    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z4zjn" event={"ID":"1545e656-1004-4d5a-aee1-7bf87e8017b2","Type":"ContainerStarted","Data":"5feed534822e0c2f230e6bda65e3c56ddabdd4585fd60881ba3b17773f14667a"}
Jul 28 20:09:05.567313 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.567292    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:09:05.567696 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.567673    2558 generic.go:356] "Generic (PLEG): container finished" podID="85d75f9d-263e-430c-9493-429d83d28190" containerID="32ff990152fcd06acb280b919f0780d01c060ad716985428c7152b6c762110d4" exitCode=1
Jul 28 20:09:05.567777 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.567755    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx" event={"ID":"85d75f9d-263e-430c-9493-429d83d28190","Type":"ContainerStarted","Data":"876fa8c4e8e8cb12e327b48a34f6c7fdc6469331f25bc50cd3d89e5d49a70a49"}
Jul 28 20:09:05.567872 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.567852    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx" event={"ID":"85d75f9d-263e-430c-9493-429d83d28190","Type":"ContainerDied","Data":"32ff990152fcd06acb280b919f0780d01c060ad716985428c7152b6c762110d4"}
Jul 28 20:09:05.567933 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.567917    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx" event={"ID":"85d75f9d-263e-430c-9493-429d83d28190","Type":"ContainerStarted","Data":"4ab85cfd469431b29765e118b060ee12b071b6e857d5779e08646b204f4f6600"}
Jul 28 20:09:05.569606 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.569582    2558 generic.go:356] "Generic (PLEG): container finished" podID="173e52b7-2fc0-4c1a-8036-96d743ff8dd1" containerID="2ebabf98ae47937ef05fa8a52cdddc7cbedc977cc765396bcf53624b85340d41" exitCode=0
Jul 28 20:09:05.569715 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.569665    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrgg9" event={"ID":"173e52b7-2fc0-4c1a-8036-96d743ff8dd1","Type":"ContainerDied","Data":"2ebabf98ae47937ef05fa8a52cdddc7cbedc977cc765396bcf53624b85340d41"}
Jul 28 20:09:05.571262 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.571180    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7hg4x" event={"ID":"98a9c952-fdae-4bdd-9944-f68a30c41f83","Type":"ContainerStarted","Data":"aef6482cc3cd880b5059b98111e7fe26dad395310ce7ff5907e341330180f5b2"}
Jul 28 20:09:05.572657 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.572628    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w628c" event={"ID":"2692fbe8-7cff-4492-99fc-a18edc6b24fa","Type":"ContainerStarted","Data":"760179206baefc241f1df1a13530f9c1ebb7cd67823f325eec72b5e7e17760bd"}
Jul 28 20:09:05.574182 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.574156    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b72f2" event={"ID":"b2678ad9-8742-49c3-98f7-7c22b69edb12","Type":"ContainerStarted","Data":"85d309dcaacd005f00f3f503dd1e20506f6df7ab9df7d488914fac9d4fbe0bd9"}
Jul 28 20:09:05.575818 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.575798    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp" event={"ID":"8e9073fe-c018-418c-8a12-d53c52a1beee","Type":"ContainerStarted","Data":"fa85438a248af93b38f54c51e293483b596fb7f650609b2d58e3183c74e6777c"}
Jul 28 20:09:05.615654 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.615390    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zhhzv" podStartSLOduration=3.06371664 podStartE2EDuration="21.615371665s" totalImagesPullingTime="18.551655025s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:05.596812924 +0000 UTC m=+22.838906146" watchObservedRunningTime="2026-07-28 20:09:05.615371665 +0000 UTC m=+22.857464887"
Jul 28 20:09:05.616156 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.616013    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-z4zjn" podStartSLOduration=3.062535104 podStartE2EDuration="21.615999908s" totalImagesPullingTime="18.553464804s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:05.615656712 +0000 UTC m=+22.857749936" watchObservedRunningTime="2026-07-28 20:09:05.615999908 +0000 UTC m=+22.858093135"
Jul 28 20:09:05.633606 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.633540    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7hg4x" podStartSLOduration=3.149616769 podStartE2EDuration="21.633522714s" totalImagesPullingTime="18.483905945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:05.633343738 +0000 UTC m=+22.875436960" watchObservedRunningTime="2026-07-28 20:09:05.633522714 +0000 UTC m=+22.875615936"
Jul 28 20:09:05.653354 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.653302    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-b72f2" podStartSLOduration=2.8550912459999997 podStartE2EDuration="21.653281994s" totalImagesPullingTime="18.798190748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:05.652655446 +0000 UTC m=+22.894748668" watchObservedRunningTime="2026-07-28 20:09:05.653281994 +0000 UTC m=+22.895375219"
Jul 28 20:09:05.690685 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:05.690641    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w628c" podStartSLOduration=3.210597739 podStartE2EDuration="21.690622424s" totalImagesPullingTime="18.480024685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:05.690594479 +0000 UTC m=+22.932687701" watchObservedRunningTime="2026-07-28 20:09:05.690622424 +0000 UTC m=+22.932715648"
Jul 28 20:09:06.334743 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:06.334704    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:06.335025 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:06.334755    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:06.335025 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:06.334894    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:06.335146 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:06.335036    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:06.583829 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:06.583795    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:09:06.584500 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:06.584320    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx" event={"ID":"85d75f9d-263e-430c-9493-429d83d28190","Type":"ContainerStarted","Data":"99fc4b31290eae21218c19e3e7457c98711dbeb17ce81c6a410a2368f84608f6"}
Jul 28 20:09:06.584590 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:06.584538    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx" event={"ID":"85d75f9d-263e-430c-9493-429d83d28190","Type":"ContainerStarted","Data":"b44164449d875865411da5cfcc8b79adac2315566e39c573c8846d3ea2c953d6"}
Jul 28 20:09:06.584648 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:06.584612    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx" event={"ID":"85d75f9d-263e-430c-9493-429d83d28190","Type":"ContainerStarted","Data":"adc06196b0bf937b8300a872520b9bfd922129323794b9b8b8cd0b47280830b1"}
Jul 28 20:09:06.708736 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:06.708703    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 20:09:07.192107 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:07.191965    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T20:09:06.708729462Z","UUID":"dccda350-25ee-4c6d-a8bc-295be09e5e44","Handler":null,"Name":"","Endpoint":""}
Jul 28 20:09:07.194164 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:07.194118    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 20:09:07.194164 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:07.194173    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 20:09:07.334324 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:07.334290    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:07.334504 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:07.334424    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:07.589428 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:07.589392    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dfchd" event={"ID":"fe05f34c-5651-4eb1-ab1a-cc871b2bfcc5","Type":"ContainerStarted","Data":"f551a42e249cd7ce5a41d141ea3c2e3aadec3c19b6a35aae7200f84a7270a85c"}
Jul 28 20:09:07.592435 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:07.592394    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp" event={"ID":"8e9073fe-c018-418c-8a12-d53c52a1beee","Type":"ContainerStarted","Data":"88fcd18b9963244d8d8c6dd7d627e0b8bc6736a140c41a43be0c8730b25951d0"}
Jul 28 20:09:07.604390 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:07.604331    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dfchd" podStartSLOduration=5.051865656 podStartE2EDuration="23.604313927s" totalImagesPullingTime="18.552448271s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:07.603983285 +0000 UTC m=+24.846076510" watchObservedRunningTime="2026-07-28 20:09:07.604313927 +0000 UTC m=+24.846407151"
Jul 28 20:09:08.335056 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:08.334797    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:08.335252 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:08.334909    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:08.335252 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:08.335178    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:08.335252 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:08.335214    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:08.598323 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:08.598226    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp" event={"ID":"8e9073fe-c018-418c-8a12-d53c52a1beee","Type":"ContainerStarted","Data":"e37f12ac159abf310a9967110122a2015529afd83d79019dfdd252f54ee8452e"}
Jul 28 20:09:08.602068 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:08.602039    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:09:08.602596 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:08.602547    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx" event={"ID":"85d75f9d-263e-430c-9493-429d83d28190","Type":"ContainerStarted","Data":"ee95e5eb7259558818e37181f563434f3e5ec6fcd466e669d7bbc9d9ce813dad"}
Jul 28 20:09:08.621134 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:08.621064    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thrlp" podStartSLOduration=3.929507102 podStartE2EDuration="24.6210368s" totalImagesPullingTime="20.691529698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:08:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:08.619629524 +0000 UTC m=+25.861722746" watchObservedRunningTime="2026-07-28 20:09:08.6210368 +0000 UTC m=+25.863130024"
Jul 28 20:09:09.127966 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:09.127921    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w628c"
Jul 28 20:09:09.128667 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:09.128639    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w628c"
Jul 28 20:09:09.334500 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:09.334461    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:09.334722 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:09.334617    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:09.604873 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:09.604838    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w628c"
Jul 28 20:09:09.605423 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:09.605345    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w628c"
Jul 28 20:09:10.334739 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:10.334517    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:10.334890 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:10.334524    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:10.334890 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:10.334852    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:10.334890 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:10.334873    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:10.610283 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:10.610251    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:09:10.611088 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:10.610659    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx" event={"ID":"85d75f9d-263e-430c-9493-429d83d28190","Type":"ContainerStarted","Data":"03ed24a2f68740d3f60f3debe59038aec6fd63ffa7f4f162cf6ff1d95be6eb1b"}
Jul 28 20:09:10.611088 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:10.610992    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:09:10.611088 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:10.611055    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:09:10.611252 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:10.611207    2558 scope.go:117] "RemoveContainer" containerID="32ff990152fcd06acb280b919f0780d01c060ad716985428c7152b6c762110d4"
Jul 28 20:09:10.612843 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:10.612820    2558 generic.go:356] "Generic (PLEG): container finished" podID="173e52b7-2fc0-4c1a-8036-96d743ff8dd1" containerID="5ae42c71da368a78c7ea42f54a978da001e63d7d5767df9debb2d9d9bde9fd07" exitCode=0
Jul 28 20:09:10.612989 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:10.612912    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrgg9" event={"ID":"173e52b7-2fc0-4c1a-8036-96d743ff8dd1","Type":"ContainerDied","Data":"5ae42c71da368a78c7ea42f54a978da001e63d7d5767df9debb2d9d9bde9fd07"}
Jul 28 20:09:10.629877 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:10.629849    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:09:11.334377 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:11.334345    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:11.334503 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:11.334464    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:11.618867 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:11.618833    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:09:11.619367 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:11.619202    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx" event={"ID":"85d75f9d-263e-430c-9493-429d83d28190","Type":"ContainerStarted","Data":"3870eea5f73b77b0624c93384e1e2695fff815a44dd7a93d8ecd45d953e36b0e"}
Jul 28 20:09:11.619583 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:11.619534    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:09:11.621649 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:11.621616    2558 generic.go:356] "Generic (PLEG): container finished" podID="173e52b7-2fc0-4c1a-8036-96d743ff8dd1" containerID="cf6de50689d9e92d9dc4a81ca80dd512535b96c01c9c11eb6c22e2ad158e8cd7" exitCode=0
Jul 28 20:09:11.621765 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:11.621676    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrgg9" event={"ID":"173e52b7-2fc0-4c1a-8036-96d743ff8dd1","Type":"ContainerDied","Data":"cf6de50689d9e92d9dc4a81ca80dd512535b96c01c9c11eb6c22e2ad158e8cd7"}
Jul 28 20:09:11.636266 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:11.636238    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:09:11.651712 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:11.651660    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx" podStartSLOduration=9.066870677 podStartE2EDuration="27.651644622s" totalImagesPullingTime="18.584773945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:11.650273389 +0000 UTC m=+28.892366611" watchObservedRunningTime="2026-07-28 20:09:11.651644622 +0000 UTC m=+28.893737882"
Jul 28 20:09:12.334785 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:12.334754    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:12.334891 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:12.334758    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:12.334926 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:12.334899    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:12.335015 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:12.334997    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:12.625706 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:12.625673    2558 generic.go:356] "Generic (PLEG): container finished" podID="173e52b7-2fc0-4c1a-8036-96d743ff8dd1" containerID="17446da1e21a85cd2546a7ef3993376100a8f4029d562e82ef27ef9a5151f5d0" exitCode=0
Jul 28 20:09:12.626117 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:12.625763    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrgg9" event={"ID":"173e52b7-2fc0-4c1a-8036-96d743ff8dd1","Type":"ContainerDied","Data":"17446da1e21a85cd2546a7ef3993376100a8f4029d562e82ef27ef9a5151f5d0"}
Jul 28 20:09:13.335429 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:13.335385    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:13.335619 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:13.335499    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:14.334317 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:14.334284    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:14.334317 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:14.334308    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:14.334854 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:14.334419    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:14.334854 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:14.334525    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:15.334411 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:15.334367    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:15.334885 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:15.334503    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:16.334820 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:16.334777    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:16.335284 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:16.334781    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:16.335284 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:16.334896    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:16.335284 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:16.335013    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:16.894096 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:16.894058    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:16.894292 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:16.894124    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs\") pod \"network-metrics-daemon-pghh9\" (UID: \"232db419-1a78-4e78-a6fe-da6c6bc7e72f\") " pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:16.894292 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:16.894220    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:09:16.894292 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:16.894229    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:09:16.894292 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:16.894273    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs podName:232db419-1a78-4e78-a6fe-da6c6bc7e72f nodeName:}" failed. No retries permitted until 2026-07-28 20:09:48.89425963 +0000 UTC m=+66.136352831 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs") pod "network-metrics-daemon-pghh9" (UID: "232db419-1a78-4e78-a6fe-da6c6bc7e72f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:09:16.894432 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:16.894298    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret podName:e1b683c6-ec1a-4f37-b354-3a366d2bd899 nodeName:}" failed. No retries permitted until 2026-07-28 20:09:48.894278931 +0000 UTC m=+66.136372151 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret") pod "global-pull-secret-syncer-zmkbq" (UID: "e1b683c6-ec1a-4f37-b354-3a366d2bd899") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:09:16.994435 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:16.994391    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdwjr\" (UniqueName: \"kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr\") pod \"network-check-target-q9m9r\" (UID: \"8db6484b-7821-440f-b467-4aa1721dbd50\") " pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:16.994629 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:16.994584    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:09:16.994629 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:16.994605    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:09:16.994629 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:16.994616    2558 projected.go:196] Error preparing data for projected volume kube-api-access-qdwjr for pod openshift-network-diagnostics/network-check-target-q9m9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:09:16.994738 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:16.994671    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr podName:8db6484b-7821-440f-b467-4aa1721dbd50 nodeName:}" failed. No retries permitted until 2026-07-28 20:09:48.994656667 +0000 UTC m=+66.236749867 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qdwjr" (UniqueName: "kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr") pod "network-check-target-q9m9r" (UID: "8db6484b-7821-440f-b467-4aa1721dbd50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:09:17.334871 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:17.334589    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:17.335266 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:17.334995    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:17.640785 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:17.640752    2558 generic.go:356] "Generic (PLEG): container finished" podID="173e52b7-2fc0-4c1a-8036-96d743ff8dd1" containerID="51986c822bcd1442b6bcbc3bc126dc0bcbbfb358407ce73b7a4f62a5db6f0088" exitCode=0
Jul 28 20:09:17.640785 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:17.640788    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrgg9" event={"ID":"173e52b7-2fc0-4c1a-8036-96d743ff8dd1","Type":"ContainerDied","Data":"51986c822bcd1442b6bcbc3bc126dc0bcbbfb358407ce73b7a4f62a5db6f0088"}
Jul 28 20:09:18.334337 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:18.334301    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:18.334656 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:18.334301    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:18.334656 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:18.334432    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:18.334656 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:18.334484    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:18.646601 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:18.646496    2558 generic.go:356] "Generic (PLEG): container finished" podID="173e52b7-2fc0-4c1a-8036-96d743ff8dd1" containerID="dc615dfd214b101b746a2e7f39cb5eee21cd4e0d314e8e31f6943da23678bdae" exitCode=0
Jul 28 20:09:18.646601 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:18.646542    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrgg9" event={"ID":"173e52b7-2fc0-4c1a-8036-96d743ff8dd1","Type":"ContainerDied","Data":"dc615dfd214b101b746a2e7f39cb5eee21cd4e0d314e8e31f6943da23678bdae"}
Jul 28 20:09:19.334327 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:19.334279    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:19.334527 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:19.334386    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:19.652698 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:19.652618    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrgg9" event={"ID":"173e52b7-2fc0-4c1a-8036-96d743ff8dd1","Type":"ContainerStarted","Data":"cc93ad399edbbe7d8f816c1eed1bded5424d7a8a124caeafe2f882d65b106f3b"}
Jul 28 20:09:19.679333 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:19.679281    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xrgg9" podStartSLOduration=7.117491904 podStartE2EDuration="35.679264285s" totalImagesPullingTime="28.561772381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:08:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:19.678843638 +0000 UTC m=+36.920936861" watchObservedRunningTime="2026-07-28 20:09:19.679264285 +0000 UTC m=+36.921357508"
Jul 28 20:09:20.333959 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:20.333916    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:20.334146 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:20.334044    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:20.334146 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:20.334092    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:20.334218 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:20.334184    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:21.334602 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:21.334549    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:21.335040 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:21.334678    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:22.334886 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:22.334843    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:22.335302 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:22.334850    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:22.335302 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:22.334982    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:22.335302 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:22.335033    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:23.335633 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:23.335595    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:23.336005 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:23.335690    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:24.334002 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:24.333965    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:24.334002 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:24.333990    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:24.334213 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:24.334081    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:24.334213 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:24.334203    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:25.335103 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:25.335067    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:25.335589 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:25.335177    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:26.334287 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:26.334250    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:26.334287 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:26.334269    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:26.334499 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:26.334361    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:26.334499 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:26.334472    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:27.334062 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:27.333995    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:27.334574 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:27.334147    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:28.334400 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:28.334357    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:28.334400 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:28.334400    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:28.335099 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:28.334488    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:28.335099 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:28.334630    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:28.417548 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:28.417504    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zmkbq"]
Jul 28 20:09:28.417815 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:28.417682    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q9m9r"]
Jul 28 20:09:28.417891 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:28.417827    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:28.418394 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:28.418043    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:28.419642 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:28.419615    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pghh9"]
Jul 28 20:09:28.673790 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:28.673760    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:28.673963 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:28.673760    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:28.673963 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:28.673895    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:28.673963 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:28.673945    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:30.334220 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:30.334187    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:30.334688 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:30.334187    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:30.334688 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:30.334307    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:30.334688 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:30.334399    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:30.334688 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:30.334188    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:30.334688 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:30.334508    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:32.334358 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:32.334167    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:32.334779 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:32.334196    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:32.334779 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:32.334444    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pghh9" podUID="232db419-1a78-4e78-a6fe-da6c6bc7e72f"
Jul 28 20:09:32.334779 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:32.334209    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:32.334779 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:32.334501    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9m9r" podUID="8db6484b-7821-440f-b467-4aa1721dbd50"
Jul 28 20:09:32.334779 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:32.334603    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmkbq" podUID="e1b683c6-ec1a-4f37-b354-3a366d2bd899"
Jul 28 20:09:33.060199 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.060111    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-72.ec2.internal" event="NodeReady"
Jul 28 20:09:33.060337 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.060242    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 20:09:33.143834 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.143801    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nk69s"]
Jul 28 20:09:33.155764 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.155733    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lsbx7"]
Jul 28 20:09:33.155952 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.155912    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nk69s"
Jul 28 20:09:33.158335 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.158298    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 20:09:33.158471 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.158396    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 20:09:33.158550 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.158473    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 20:09:33.158550 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.158492    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k95ng\""
Jul 28 20:09:33.169759 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.169730    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nk69s"]
Jul 28 20:09:33.169929 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.169773    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lsbx7"]
Jul 28 20:09:33.169929 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.169900    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.172551 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.172534    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 20:09:33.174022 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.173997    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 20:09:33.174022 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.174019    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tn5xs\""
Jul 28 20:09:33.174218 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.174193    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 20:09:33.174361 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.174343    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 20:09:33.232282 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.232247    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zl2gn"]
Jul 28 20:09:33.254436 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.254406    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zl2gn"]
Jul 28 20:09:33.254593 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.254537    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.257138 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.257112    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-488rx\""
Jul 28 20:09:33.257291 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.257142    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 20:09:33.257291 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.257111    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 20:09:33.309078 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.309014    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a7a6337-4154-40f0-bc94-f66cd5da615b-crio-socket\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.309256 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.309101    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9b4e7cf-6cbf-4067-9491-f5c3b83663a0-cert\") pod \"ingress-canary-nk69s\" (UID: \"c9b4e7cf-6cbf-4067-9491-f5c3b83663a0\") " pod="openshift-ingress-canary/ingress-canary-nk69s"
Jul 28 20:09:33.309256 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.309125    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrbnx\" (UniqueName: \"kubernetes.io/projected/c9b4e7cf-6cbf-4067-9491-f5c3b83663a0-kube-api-access-jrbnx\") pod \"ingress-canary-nk69s\" (UID: \"c9b4e7cf-6cbf-4067-9491-f5c3b83663a0\") " pod="openshift-ingress-canary/ingress-canary-nk69s"
Jul 28 20:09:33.309256 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.309143    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl6jd\" (UniqueName: \"kubernetes.io/projected/4a7a6337-4154-40f0-bc94-f66cd5da615b-kube-api-access-kl6jd\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.309256 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.309161    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a7a6337-4154-40f0-bc94-f66cd5da615b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.309256 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.309184    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a7a6337-4154-40f0-bc94-f66cd5da615b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.309409 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.309276    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a7a6337-4154-40f0-bc94-f66cd5da615b-data-volume\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.410128 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.410031    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9b4e7cf-6cbf-4067-9491-f5c3b83663a0-cert\") pod \"ingress-canary-nk69s\" (UID: \"c9b4e7cf-6cbf-4067-9491-f5c3b83663a0\") " pod="openshift-ingress-canary/ingress-canary-nk69s"
Jul 28 20:09:33.410128 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.410075    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrbnx\" (UniqueName: \"kubernetes.io/projected/c9b4e7cf-6cbf-4067-9491-f5c3b83663a0-kube-api-access-jrbnx\") pod \"ingress-canary-nk69s\" (UID: \"c9b4e7cf-6cbf-4067-9491-f5c3b83663a0\") " pod="openshift-ingress-canary/ingress-canary-nk69s"
Jul 28 20:09:33.410128 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.410100    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl6jd\" (UniqueName: \"kubernetes.io/projected/4a7a6337-4154-40f0-bc94-f66cd5da615b-kube-api-access-kl6jd\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.410752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.410228    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a7a6337-4154-40f0-bc94-f66cd5da615b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.410752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.410281    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a7a6337-4154-40f0-bc94-f66cd5da615b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.410752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.410316    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73595785-7c8d-48c2-9d25-8b2af9520034-config-volume\") pod \"dns-default-zl2gn\" (UID: \"73595785-7c8d-48c2-9d25-8b2af9520034\") " pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.410752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.410354    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73595785-7c8d-48c2-9d25-8b2af9520034-tmp-dir\") pod \"dns-default-zl2gn\" (UID: \"73595785-7c8d-48c2-9d25-8b2af9520034\") " pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.410752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.410405    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a7a6337-4154-40f0-bc94-f66cd5da615b-data-volume\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.410752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.410431    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv6xc\" (UniqueName: \"kubernetes.io/projected/73595785-7c8d-48c2-9d25-8b2af9520034-kube-api-access-mv6xc\") pod \"dns-default-zl2gn\" (UID: \"73595785-7c8d-48c2-9d25-8b2af9520034\") " pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.410752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.410483    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73595785-7c8d-48c2-9d25-8b2af9520034-metrics-tls\") pod \"dns-default-zl2gn\" (UID: \"73595785-7c8d-48c2-9d25-8b2af9520034\") " pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.410752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.410522    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a7a6337-4154-40f0-bc94-f66cd5da615b-crio-socket\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.410752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.410717    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a7a6337-4154-40f0-bc94-f66cd5da615b-crio-socket\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.411176 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.410821    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a7a6337-4154-40f0-bc94-f66cd5da615b-data-volume\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.411176 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.411060    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a7a6337-4154-40f0-bc94-f66cd5da615b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.414375 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.414340    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a7a6337-4154-40f0-bc94-f66cd5da615b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.414439 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.414375    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9b4e7cf-6cbf-4067-9491-f5c3b83663a0-cert\") pod \"ingress-canary-nk69s\" (UID: \"c9b4e7cf-6cbf-4067-9491-f5c3b83663a0\") " pod="openshift-ingress-canary/ingress-canary-nk69s"
Jul 28 20:09:33.417779 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.417754    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl6jd\" (UniqueName: \"kubernetes.io/projected/4a7a6337-4154-40f0-bc94-f66cd5da615b-kube-api-access-kl6jd\") pod \"insights-runtime-extractor-lsbx7\" (UID: \"4a7a6337-4154-40f0-bc94-f66cd5da615b\") " pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.417918 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.417897    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrbnx\" (UniqueName: \"kubernetes.io/projected/c9b4e7cf-6cbf-4067-9491-f5c3b83663a0-kube-api-access-jrbnx\") pod \"ingress-canary-nk69s\" (UID: \"c9b4e7cf-6cbf-4067-9491-f5c3b83663a0\") " pod="openshift-ingress-canary/ingress-canary-nk69s"
Jul 28 20:09:33.466745 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.466707    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nk69s"
Jul 28 20:09:33.478884 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.478846    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lsbx7"
Jul 28 20:09:33.511532 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.511495    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73595785-7c8d-48c2-9d25-8b2af9520034-tmp-dir\") pod \"dns-default-zl2gn\" (UID: \"73595785-7c8d-48c2-9d25-8b2af9520034\") " pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.511727 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.511544    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv6xc\" (UniqueName: \"kubernetes.io/projected/73595785-7c8d-48c2-9d25-8b2af9520034-kube-api-access-mv6xc\") pod \"dns-default-zl2gn\" (UID: \"73595785-7c8d-48c2-9d25-8b2af9520034\") " pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.511727 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.511589    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73595785-7c8d-48c2-9d25-8b2af9520034-metrics-tls\") pod \"dns-default-zl2gn\" (UID: \"73595785-7c8d-48c2-9d25-8b2af9520034\") " pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.511727 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.511635    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73595785-7c8d-48c2-9d25-8b2af9520034-config-volume\") pod \"dns-default-zl2gn\" (UID: \"73595785-7c8d-48c2-9d25-8b2af9520034\") " pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.512046 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.512025    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73595785-7c8d-48c2-9d25-8b2af9520034-tmp-dir\") pod \"dns-default-zl2gn\" (UID: \"73595785-7c8d-48c2-9d25-8b2af9520034\") " pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.512265 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.512242    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73595785-7c8d-48c2-9d25-8b2af9520034-config-volume\") pod \"dns-default-zl2gn\" (UID: \"73595785-7c8d-48c2-9d25-8b2af9520034\") " pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.514672 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.514648    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73595785-7c8d-48c2-9d25-8b2af9520034-metrics-tls\") pod \"dns-default-zl2gn\" (UID: \"73595785-7c8d-48c2-9d25-8b2af9520034\") " pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.522018 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.521992    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv6xc\" (UniqueName: \"kubernetes.io/projected/73595785-7c8d-48c2-9d25-8b2af9520034-kube-api-access-mv6xc\") pod \"dns-default-zl2gn\" (UID: \"73595785-7c8d-48c2-9d25-8b2af9520034\") " pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.564537 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.564503    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:33.701686 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.701652    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lsbx7"]
Jul 28 20:09:33.705093 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.705028    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nk69s"]
Jul 28 20:09:33.709725 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:33.708744    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a7a6337_4154_40f0_bc94_f66cd5da615b.slice/crio-a186dc27ad10572ab386f48815567f73beccf917218fec8b91d4aef1cc3f02a1 WatchSource:0}: Error finding container a186dc27ad10572ab386f48815567f73beccf917218fec8b91d4aef1cc3f02a1: Status 404 returned error can't find the container with id a186dc27ad10572ab386f48815567f73beccf917218fec8b91d4aef1cc3f02a1
Jul 28 20:09:33.710139 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:33.710088    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9b4e7cf_6cbf_4067_9491_f5c3b83663a0.slice/crio-84eff215455a04cf92af2c642c83adf7bb69221ed8142f8c601a676e878bb46d WatchSource:0}: Error finding container 84eff215455a04cf92af2c642c83adf7bb69221ed8142f8c601a676e878bb46d: Status 404 returned error can't find the container with id 84eff215455a04cf92af2c642c83adf7bb69221ed8142f8c601a676e878bb46d
Jul 28 20:09:33.750733 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:33.750701    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zl2gn"]
Jul 28 20:09:33.753808 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:33.753779    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73595785_7c8d_48c2_9d25_8b2af9520034.slice/crio-ce1c3b7640cd3c49686b54511ed2895abd93007a9e15fbe5e2ec8da290bd6036 WatchSource:0}: Error finding container ce1c3b7640cd3c49686b54511ed2895abd93007a9e15fbe5e2ec8da290bd6036: Status 404 returned error can't find the container with id ce1c3b7640cd3c49686b54511ed2895abd93007a9e15fbe5e2ec8da290bd6036
Jul 28 20:09:34.334702 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:34.334667    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:34.335062 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:34.334666    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:34.335200 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:34.334672    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:34.338718 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:34.338682    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 20:09:34.339053 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:34.339019    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 20:09:34.339432 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:34.339411    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-82krx\""
Jul 28 20:09:34.340049 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:34.339584    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 20:09:34.340049 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:34.339802    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 20:09:34.340049 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:34.339969    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sp59p\""
Jul 28 20:09:34.691241 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:34.691148    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zl2gn" event={"ID":"73595785-7c8d-48c2-9d25-8b2af9520034","Type":"ContainerStarted","Data":"ce1c3b7640cd3c49686b54511ed2895abd93007a9e15fbe5e2ec8da290bd6036"}
Jul 28 20:09:34.692994 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:34.692963    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lsbx7" event={"ID":"4a7a6337-4154-40f0-bc94-f66cd5da615b","Type":"ContainerStarted","Data":"a186dc27ad10572ab386f48815567f73beccf917218fec8b91d4aef1cc3f02a1"}
Jul 28 20:09:34.695048 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:34.695023    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nk69s" event={"ID":"c9b4e7cf-6cbf-4067-9491-f5c3b83663a0","Type":"ContainerStarted","Data":"84eff215455a04cf92af2c642c83adf7bb69221ed8142f8c601a676e878bb46d"}
Jul 28 20:09:38.607453 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.607272    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f46b6dcfb-8p2mw"]
Jul 28 20:09:38.632973 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.632931    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f46b6dcfb-8p2mw"]
Jul 28 20:09:38.633165 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.633085    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.635997 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.635973    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 20:09:38.635997 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.635985    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 20:09:38.635997 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.635975    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 20:09:38.636238 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.635975    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 20:09:38.636864 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.636843    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9zrj5\""
Jul 28 20:09:38.636962 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.636878    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 20:09:38.637214 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.637198    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 20:09:38.637214 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.637208    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 20:09:38.641196 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.641171    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 20:09:38.652155 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.652123    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-service-ca\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.652336 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.652208    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-trusted-ca-bundle\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.652336 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.652254    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-oauth-serving-cert\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.652336 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.652278    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnlhq\" (UniqueName: \"kubernetes.io/projected/9be6a081-53cb-449d-a9d7-ecfa5a37a683-kube-api-access-pnlhq\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.652467 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.652332    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-serving-cert\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.652467 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.652372    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-oauth-config\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.652467 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.652400    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-config\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.753351 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.753316    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-service-ca\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.753474 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.753460    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-trusted-ca-bundle\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.753519 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.753497    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-oauth-serving-cert\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.753583 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.753519    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnlhq\" (UniqueName: \"kubernetes.io/projected/9be6a081-53cb-449d-a9d7-ecfa5a37a683-kube-api-access-pnlhq\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.753583 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.753542    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-serving-cert\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.753583 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.753575    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-oauth-config\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.753743 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.753599    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-config\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.754150 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.754116    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-service-ca\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.754248 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.754212    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-oauth-serving-cert\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.754367 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.754331    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-trusted-ca-bundle\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.754664 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.754643    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-config\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.757113 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.757088    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-serving-cert\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.757208 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.757091    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-oauth-config\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.761109 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.761087    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnlhq\" (UniqueName: \"kubernetes.io/projected/9be6a081-53cb-449d-a9d7-ecfa5a37a683-kube-api-access-pnlhq\") pod \"console-5f46b6dcfb-8p2mw\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") " pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:38.943762 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:38.943670    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:39.256762 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.256728    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-g9fxr"]
Jul 28 20:09:39.258839 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.258812    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-g9fxr"
Jul 28 20:09:39.261260 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.261238    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-wvpd6\""
Jul 28 20:09:39.261391 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.261282    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 20:09:39.266359 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.266219    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-g9fxr"]
Jul 28 20:09:39.360309 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.360269    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d58be913-bc73-4252-a974-5ebffbb9485a-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-g9fxr\" (UID: \"d58be913-bc73-4252-a974-5ebffbb9485a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-g9fxr"
Jul 28 20:09:39.466527 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.466017    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d58be913-bc73-4252-a974-5ebffbb9485a-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-g9fxr\" (UID: \"d58be913-bc73-4252-a974-5ebffbb9485a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-g9fxr"
Jul 28 20:09:39.468662 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.468634    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d58be913-bc73-4252-a974-5ebffbb9485a-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-g9fxr\" (UID: \"d58be913-bc73-4252-a974-5ebffbb9485a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-g9fxr"
Jul 28 20:09:39.534016 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.533983    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f46b6dcfb-8p2mw"]
Jul 28 20:09:39.569369 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.569335    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-g9fxr"
Jul 28 20:09:39.703460 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.703425    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-g9fxr"]
Jul 28 20:09:39.706681 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:39.706656    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58be913_bc73_4252_a974_5ebffbb9485a.slice/crio-67fbc02102b89a297377a5421a9619ee14f9e7aea63e7403dd227ad67258f228 WatchSource:0}: Error finding container 67fbc02102b89a297377a5421a9619ee14f9e7aea63e7403dd227ad67258f228: Status 404 returned error can't find the container with id 67fbc02102b89a297377a5421a9619ee14f9e7aea63e7403dd227ad67258f228
Jul 28 20:09:39.709685 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.709654    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lsbx7" event={"ID":"4a7a6337-4154-40f0-bc94-f66cd5da615b","Type":"ContainerStarted","Data":"966bad55a6b557c6c4cfe0192f5ef7a9da314f5491059911759db458cbe9bb2b"}
Jul 28 20:09:39.711500 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.711477    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zl2gn" event={"ID":"73595785-7c8d-48c2-9d25-8b2af9520034","Type":"ContainerStarted","Data":"a975b8a5d86422dbb15f0100f3cc969017711b8414f55571c33d4be9889af7f9"}
Jul 28 20:09:39.711643 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.711522    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zl2gn" event={"ID":"73595785-7c8d-48c2-9d25-8b2af9520034","Type":"ContainerStarted","Data":"2ef37fc78ee4051218c243e6e5909d97dba33ab8de749a35a51a43b49d0abb4a"}
Jul 28 20:09:39.711643 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.711602    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:39.713016 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.712990    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nk69s" event={"ID":"c9b4e7cf-6cbf-4067-9491-f5c3b83663a0","Type":"ContainerStarted","Data":"4bcac5c9d044a312ee8dde2b7142e8314e2d24a907f005f4f31391f736e7e3ad"}
Jul 28 20:09:39.714159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.714095    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f46b6dcfb-8p2mw" event={"ID":"9be6a081-53cb-449d-a9d7-ecfa5a37a683","Type":"ContainerStarted","Data":"989e44579e42500937785866bf96574a67e1f5b047a3641984222867cde5429a"}
Jul 28 20:09:39.729357 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.729293    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zl2gn" podStartSLOduration=1.106560052 podStartE2EDuration="6.729274338s" totalImagesPullingTime="5.622714286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:39.728265528 +0000 UTC m=+56.970358751" watchObservedRunningTime="2026-07-28 20:09:39.729274338 +0000 UTC m=+56.971367539"
Jul 28 20:09:39.744269 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:39.744214    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nk69s" podStartSLOduration=1.077665923 podStartE2EDuration="6.744196131s" totalImagesPullingTime="5.666530208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:39.743700467 +0000 UTC m=+56.985793691" watchObservedRunningTime="2026-07-28 20:09:39.744196131 +0000 UTC m=+56.986289368"
Jul 28 20:09:40.720161 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:40.720028    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-g9fxr" event={"ID":"d58be913-bc73-4252-a974-5ebffbb9485a","Type":"ContainerStarted","Data":"67fbc02102b89a297377a5421a9619ee14f9e7aea63e7403dd227ad67258f228"}
Jul 28 20:09:43.640625 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:43.640593    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-czcmx"
Jul 28 20:09:44.732854 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:44.732815    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f46b6dcfb-8p2mw" event={"ID":"9be6a081-53cb-449d-a9d7-ecfa5a37a683","Type":"ContainerStarted","Data":"ae1b7cbd3357f2f1fd5c27e7dde6d4db089d902dbed7482d4f2263b1ac6bc8cd"}
Jul 28 20:09:44.734344 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:44.734319    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-g9fxr" event={"ID":"d58be913-bc73-4252-a974-5ebffbb9485a","Type":"ContainerStarted","Data":"a68bf3511a087d442f6e6fa3eeb6089e5bae45955e9225e7974ba71f0c3a4e57"}
Jul 28 20:09:44.734494 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:44.734478    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-g9fxr"
Jul 28 20:09:44.736342 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:44.736314    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lsbx7" event={"ID":"4a7a6337-4154-40f0-bc94-f66cd5da615b","Type":"ContainerStarted","Data":"177cf5bafc17aab47dbcb85bf4adbab7158d27398c78d8343fc7523a6866dc06"}
Jul 28 20:09:44.739691 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:44.739670    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-g9fxr"
Jul 28 20:09:44.750627 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:44.750533    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f46b6dcfb-8p2mw" podStartSLOduration=1.760262273 podStartE2EDuration="6.750504667s" totalImagesPullingTime="4.990242394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:44.749063376 +0000 UTC m=+61.991156599" watchObservedRunningTime="2026-07-28 20:09:44.750504667 +0000 UTC m=+61.992597890"
Jul 28 20:09:44.763951 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:44.763843    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-g9fxr" podStartSLOduration=0.962190007 podStartE2EDuration="5.763823086s" totalImagesPullingTime="4.801633079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:44.762795291 +0000 UTC m=+62.004888496" watchObservedRunningTime="2026-07-28 20:09:44.763823086 +0000 UTC m=+62.005916287"
Jul 28 20:09:45.323370 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.323333    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"]
Jul 28 20:09:45.325738 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.325714    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.328179 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.328156    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 20:09:45.328486 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.328468    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 28 20:09:45.328595 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.328567    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 28 20:09:45.329242 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.329225    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 20:09:45.329328 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.329251    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 20:09:45.329433 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.329414    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-tjhqc\""
Jul 28 20:09:45.338866 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.338838    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"]
Jul 28 20:09:45.418890 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.418845    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a87ca62-5860-4bde-80d9-600f1b9172d1-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-wg7mf\" (UID: \"1a87ca62-5860-4bde-80d9-600f1b9172d1\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.419084 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.418903    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a87ca62-5860-4bde-80d9-600f1b9172d1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-wg7mf\" (UID: \"1a87ca62-5860-4bde-80d9-600f1b9172d1\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.419084 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.418935    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w4r5\" (UniqueName: \"kubernetes.io/projected/1a87ca62-5860-4bde-80d9-600f1b9172d1-kube-api-access-6w4r5\") pod \"prometheus-operator-84d69f794d-wg7mf\" (UID: \"1a87ca62-5860-4bde-80d9-600f1b9172d1\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.419084 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.419037    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a87ca62-5860-4bde-80d9-600f1b9172d1-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-wg7mf\" (UID: \"1a87ca62-5860-4bde-80d9-600f1b9172d1\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.520087 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.520042    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a87ca62-5860-4bde-80d9-600f1b9172d1-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-wg7mf\" (UID: \"1a87ca62-5860-4bde-80d9-600f1b9172d1\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.520289 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.520136    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a87ca62-5860-4bde-80d9-600f1b9172d1-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-wg7mf\" (UID: \"1a87ca62-5860-4bde-80d9-600f1b9172d1\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.520289 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.520167    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a87ca62-5860-4bde-80d9-600f1b9172d1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-wg7mf\" (UID: \"1a87ca62-5860-4bde-80d9-600f1b9172d1\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.520289 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.520193    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w4r5\" (UniqueName: \"kubernetes.io/projected/1a87ca62-5860-4bde-80d9-600f1b9172d1-kube-api-access-6w4r5\") pod \"prometheus-operator-84d69f794d-wg7mf\" (UID: \"1a87ca62-5860-4bde-80d9-600f1b9172d1\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.520899 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.520867    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a87ca62-5860-4bde-80d9-600f1b9172d1-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-wg7mf\" (UID: \"1a87ca62-5860-4bde-80d9-600f1b9172d1\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.523109 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.523074    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a87ca62-5860-4bde-80d9-600f1b9172d1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-wg7mf\" (UID: \"1a87ca62-5860-4bde-80d9-600f1b9172d1\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.523220 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.523189    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a87ca62-5860-4bde-80d9-600f1b9172d1-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-wg7mf\" (UID: \"1a87ca62-5860-4bde-80d9-600f1b9172d1\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.528814 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.528760    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w4r5\" (UniqueName: \"kubernetes.io/projected/1a87ca62-5860-4bde-80d9-600f1b9172d1-kube-api-access-6w4r5\") pod \"prometheus-operator-84d69f794d-wg7mf\" (UID: \"1a87ca62-5860-4bde-80d9-600f1b9172d1\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.637527 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.637449    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"
Jul 28 20:09:45.785168 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:45.785132    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-wg7mf"]
Jul 28 20:09:45.788925 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:45.788889    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a87ca62_5860_4bde_80d9_600f1b9172d1.slice/crio-5e4c1e0ede2ddd0c0cb6eedb16f05f1a96f04e5b50dbf0b3243b10f78fd99272 WatchSource:0}: Error finding container 5e4c1e0ede2ddd0c0cb6eedb16f05f1a96f04e5b50dbf0b3243b10f78fd99272: Status 404 returned error can't find the container with id 5e4c1e0ede2ddd0c0cb6eedb16f05f1a96f04e5b50dbf0b3243b10f78fd99272
Jul 28 20:09:46.746000 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:46.745962    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf" event={"ID":"1a87ca62-5860-4bde-80d9-600f1b9172d1","Type":"ContainerStarted","Data":"5e4c1e0ede2ddd0c0cb6eedb16f05f1a96f04e5b50dbf0b3243b10f78fd99272"}
Jul 28 20:09:48.944228 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:48.944184    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:48.944668 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:48.944302    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:09:48.945693 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:48.945666    2558 patch_prober.go:28] interesting pod/console-5f46b6dcfb-8p2mw container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body=
Jul 28 20:09:48.945802 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:48.945712    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5f46b6dcfb-8p2mw" podUID="9be6a081-53cb-449d-a9d7-ecfa5a37a683" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused"
Jul 28 20:09:48.951352 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:48.951331    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs\") pod \"network-metrics-daemon-pghh9\" (UID: \"232db419-1a78-4e78-a6fe-da6c6bc7e72f\") " pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:48.951412 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:48.951372    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:48.953739 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:48.953717    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 20:09:48.954523 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:48.954506    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 20:09:48.964541 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:48.964513    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232db419-1a78-4e78-a6fe-da6c6bc7e72f-metrics-certs\") pod \"network-metrics-daemon-pghh9\" (UID: \"232db419-1a78-4e78-a6fe-da6c6bc7e72f\") " pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:48.964652 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:48.964543    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1b683c6-ec1a-4f37-b354-3a366d2bd899-original-pull-secret\") pod \"global-pull-secret-syncer-zmkbq\" (UID: \"e1b683c6-ec1a-4f37-b354-3a366d2bd899\") " pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:49.052318 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.052291    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdwjr\" (UniqueName: \"kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr\") pod \"network-check-target-q9m9r\" (UID: \"8db6484b-7821-440f-b467-4aa1721dbd50\") " pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:49.054240 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.054214    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmkbq"
Jul 28 20:09:49.056887 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.056770    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 20:09:49.065770 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.065750    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 20:09:49.066777 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.066749    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-82krx\""
Jul 28 20:09:49.074857 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.074827    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pghh9"
Jul 28 20:09:49.076934 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.076812    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdwjr\" (UniqueName: \"kubernetes.io/projected/8db6484b-7821-440f-b467-4aa1721dbd50-kube-api-access-qdwjr\") pod \"network-check-target-q9m9r\" (UID: \"8db6484b-7821-440f-b467-4aa1721dbd50\") " pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:49.222053 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.221969    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zmkbq"]
Jul 28 20:09:49.224880 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:49.224851    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b683c6_ec1a_4f37_b354_3a366d2bd899.slice/crio-3c277c298a915fbcbaf9e7fb79039591e4a6df263e5f3e43fbd13e5621fc27b7 WatchSource:0}: Error finding container 3c277c298a915fbcbaf9e7fb79039591e4a6df263e5f3e43fbd13e5621fc27b7: Status 404 returned error can't find the container with id 3c277c298a915fbcbaf9e7fb79039591e4a6df263e5f3e43fbd13e5621fc27b7
Jul 28 20:09:49.259099 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.258943    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pghh9"]
Jul 28 20:09:49.261521 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:49.261496    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod232db419_1a78_4e78_a6fe_da6c6bc7e72f.slice/crio-30db1ad7d35bbf48fd6e96b40ece45c9cc7e918197e8fcded37dbb2a524f15b5 WatchSource:0}: Error finding container 30db1ad7d35bbf48fd6e96b40ece45c9cc7e918197e8fcded37dbb2a524f15b5: Status 404 returned error can't find the container with id 30db1ad7d35bbf48fd6e96b40ece45c9cc7e918197e8fcded37dbb2a524f15b5
Jul 28 20:09:49.377174 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.377144    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sp59p\""
Jul 28 20:09:49.385039 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.385007    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:49.512367 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.512333    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q9m9r"]
Jul 28 20:09:49.516199 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:49.516172    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db6484b_7821_440f_b467_4aa1721dbd50.slice/crio-d5467e68e41ddc949528d6672b34460b3e002957c5fde47a9fafa7b5b8cfcd89 WatchSource:0}: Error finding container d5467e68e41ddc949528d6672b34460b3e002957c5fde47a9fafa7b5b8cfcd89: Status 404 returned error can't find the container with id d5467e68e41ddc949528d6672b34460b3e002957c5fde47a9fafa7b5b8cfcd89
Jul 28 20:09:49.722686 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.722650    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zl2gn"
Jul 28 20:09:49.759516 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.759431    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lsbx7" event={"ID":"4a7a6337-4154-40f0-bc94-f66cd5da615b","Type":"ContainerStarted","Data":"a4ad3aa5340ad2fcddf204c1b492a5c879a466094931fc74cd391be591695985"}
Jul 28 20:09:49.760949 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.760916    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pghh9" event={"ID":"232db419-1a78-4e78-a6fe-da6c6bc7e72f","Type":"ContainerStarted","Data":"30db1ad7d35bbf48fd6e96b40ece45c9cc7e918197e8fcded37dbb2a524f15b5"}
Jul 28 20:09:49.763092 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.763014    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf" event={"ID":"1a87ca62-5860-4bde-80d9-600f1b9172d1","Type":"ContainerStarted","Data":"c6aa2bcc0b91a390ef59d6c800c2fa99d008cff654541837e6c0d9a040f946dd"}
Jul 28 20:09:49.763092 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.763065    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf" event={"ID":"1a87ca62-5860-4bde-80d9-600f1b9172d1","Type":"ContainerStarted","Data":"194a61807fa36623cc8206d10dca0e645e783c4ed5e9bab2d5f2bc61524ea5c5"}
Jul 28 20:09:49.764416 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.764382    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q9m9r" event={"ID":"8db6484b-7821-440f-b467-4aa1721dbd50","Type":"ContainerStarted","Data":"d5467e68e41ddc949528d6672b34460b3e002957c5fde47a9fafa7b5b8cfcd89"}
Jul 28 20:09:49.765632 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.765601    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zmkbq" event={"ID":"e1b683c6-ec1a-4f37-b354-3a366d2bd899","Type":"ContainerStarted","Data":"3c277c298a915fbcbaf9e7fb79039591e4a6df263e5f3e43fbd13e5621fc27b7"}
Jul 28 20:09:49.787052 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.786988    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lsbx7" podStartSLOduration=1.672802591 podStartE2EDuration="16.786962732s" totalImagesPullingTime="15.114160141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:09:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:49.785429978 +0000 UTC m=+67.027523203" watchObservedRunningTime="2026-07-28 20:09:49.786962732 +0000 UTC m=+67.029055959"
Jul 28 20:09:49.807365 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:49.807296    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-wg7mf" podStartSLOduration=1.612769821 podStartE2EDuration="4.807270101s" totalImagesPullingTime="3.19450028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:49.80586019 +0000 UTC m=+67.047953415" watchObservedRunningTime="2026-07-28 20:09:49.807270101 +0000 UTC m=+67.049363324"
Jul 28 20:09:50.772437 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:50.772140    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pghh9" event={"ID":"232db419-1a78-4e78-a6fe-da6c6bc7e72f","Type":"ContainerStarted","Data":"be35a1286cd564185d1313962274be1101bf32c0236b45534a9dbd99de34bd11"}
Jul 28 20:09:50.772437 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:50.772219    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pghh9" event={"ID":"232db419-1a78-4e78-a6fe-da6c6bc7e72f","Type":"ContainerStarted","Data":"697ce9cb30beab241115e251da9128bd4ab4b3cdbfa5d19db8f82e22cd2e1eaf"}
Jul 28 20:09:50.796341 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:50.796208    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pghh9" podStartSLOduration=65.897398807 podStartE2EDuration="1m6.796185455s" totalImagesPullingTime="898.786648ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:50.794263468 +0000 UTC m=+68.036356685" watchObservedRunningTime="2026-07-28 20:09:50.796185455 +0000 UTC m=+68.038278682"
Jul 28 20:09:51.900896 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.900100    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"]
Jul 28 20:09:51.904248 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.903848    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:51.906935 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.906844    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xj747\""
Jul 28 20:09:51.907974 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.907943    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 20:09:51.908097 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.908070    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 20:09:51.929197 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.929155    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"]
Jul 28 20:09:51.940809 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.939598    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"]
Jul 28 20:09:51.943353 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.943322    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pl5jb"]
Jul 28 20:09:51.943503 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.943417    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:51.946011 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.945849    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:51.947006 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.946985    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 20:09:51.947472 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.947301    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-j6qht\""
Jul 28 20:09:51.947472 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.947406    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 20:09:51.947760 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.947679    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 20:09:51.951122 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.950950    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 20:09:51.951122 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.950995    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 20:09:51.951508 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.951489    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w8qwr\""
Jul 28 20:09:51.951621 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.951584    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 20:09:51.975018 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.974881    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"]
Jul 28 20:09:51.975186 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.975068    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8xt4\" (UniqueName: \"kubernetes.io/projected/95abda6d-c7b6-4b65-87f6-f5a3af15f232-kube-api-access-h8xt4\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:51.975186 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.975107    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95abda6d-c7b6-4b65-87f6-f5a3af15f232-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:51.975186 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.975136    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95abda6d-c7b6-4b65-87f6-f5a3af15f232-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:51.975345 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:51.975190    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95abda6d-c7b6-4b65-87f6-f5a3af15f232-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:52.075874 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.075748    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8xt4\" (UniqueName: \"kubernetes.io/projected/95abda6d-c7b6-4b65-87f6-f5a3af15f232-kube-api-access-h8xt4\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:52.075874 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.075803    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95abda6d-c7b6-4b65-87f6-f5a3af15f232-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:52.075874 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.075826    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95abda6d-c7b6-4b65-87f6-f5a3af15f232-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:52.075874 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.075851    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-tls\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.075874 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.075875    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/124e3837-d436-4036-8940-007ce3e88ab5-sys\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.076281 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.075938    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/517464c7-0e99-4501-a815-8cf75872c95c-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.076281 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:52.076004    2558 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 28 20:09:52.076281 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:52.076080    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95abda6d-c7b6-4b65-87f6-f5a3af15f232-openshift-state-metrics-tls podName:95abda6d-c7b6-4b65-87f6-f5a3af15f232 nodeName:}" failed. No retries permitted until 2026-07-28 20:09:52.576058691 +0000 UTC m=+69.818151912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/95abda6d-c7b6-4b65-87f6-f5a3af15f232-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-s5gr8" (UID: "95abda6d-c7b6-4b65-87f6-f5a3af15f232") : secret "openshift-state-metrics-tls" not found
Jul 28 20:09:52.076281 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.076233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/124e3837-d436-4036-8940-007ce3e88ab5-metrics-client-ca\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.076281 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.076266    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg7qj\" (UniqueName: \"kubernetes.io/projected/517464c7-0e99-4501-a815-8cf75872c95c-kube-api-access-lg7qj\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.076545 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.076297    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-textfile\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.076545 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.076342    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.076545 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.076382    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/517464c7-0e99-4501-a815-8cf75872c95c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.076545 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.076417    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/124e3837-d436-4036-8940-007ce3e88ab5-root\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.076545 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.076459    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95abda6d-c7b6-4b65-87f6-f5a3af15f232-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:52.076545 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.076484    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-wtmp\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.076545 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.076508    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7ctc\" (UniqueName: \"kubernetes.io/projected/124e3837-d436-4036-8940-007ce3e88ab5-kube-api-access-v7ctc\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.076778 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.076584    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/517464c7-0e99-4501-a815-8cf75872c95c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.076778 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.076622    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-accelerators-collector-config\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.076778 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.076645    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/517464c7-0e99-4501-a815-8cf75872c95c-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.076778 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.076685    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/517464c7-0e99-4501-a815-8cf75872c95c-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.077178 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.077151    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95abda6d-c7b6-4b65-87f6-f5a3af15f232-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:52.079026 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.078989    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95abda6d-c7b6-4b65-87f6-f5a3af15f232-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:52.091647 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.091607    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8xt4\" (UniqueName: \"kubernetes.io/projected/95abda6d-c7b6-4b65-87f6-f5a3af15f232-kube-api-access-h8xt4\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:52.177792 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.177608    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/124e3837-d436-4036-8940-007ce3e88ab5-root\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.177792 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.177666    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-wtmp\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.177792 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.177689    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/124e3837-d436-4036-8940-007ce3e88ab5-root\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.177792 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.177695    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7ctc\" (UniqueName: \"kubernetes.io/projected/124e3837-d436-4036-8940-007ce3e88ab5-kube-api-access-v7ctc\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.177792 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.177774    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/517464c7-0e99-4501-a815-8cf75872c95c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.178191 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.177813    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-accelerators-collector-config\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.178191 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.177831    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-wtmp\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.178191 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.177844    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/517464c7-0e99-4501-a815-8cf75872c95c-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.178191 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.178105    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/517464c7-0e99-4501-a815-8cf75872c95c-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.178380 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.178193    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-tls\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.178380 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.178229    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/124e3837-d436-4036-8940-007ce3e88ab5-sys\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.178380 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.178262    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/517464c7-0e99-4501-a815-8cf75872c95c-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.178380 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.178273    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/517464c7-0e99-4501-a815-8cf75872c95c-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.178380 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.178296    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/124e3837-d436-4036-8940-007ce3e88ab5-metrics-client-ca\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.178380 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.178322    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg7qj\" (UniqueName: \"kubernetes.io/projected/517464c7-0e99-4501-a815-8cf75872c95c-kube-api-access-lg7qj\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.178380 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.178353    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-textfile\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.178748 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.178386    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.178748 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.178423    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/517464c7-0e99-4501-a815-8cf75872c95c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.178748 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.178505    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-accelerators-collector-config\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.178748 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.178665    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/517464c7-0e99-4501-a815-8cf75872c95c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.179114 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.179081    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-textfile\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.179114 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.179096    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/124e3837-d436-4036-8940-007ce3e88ab5-metrics-client-ca\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.179342 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.179177    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/124e3837-d436-4036-8940-007ce3e88ab5-sys\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.179342 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.179294    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/517464c7-0e99-4501-a815-8cf75872c95c-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.181809 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.181777    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/517464c7-0e99-4501-a815-8cf75872c95c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.181975 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.181878    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.182187 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.182168    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/124e3837-d436-4036-8940-007ce3e88ab5-node-exporter-tls\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.183544 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.183495    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/517464c7-0e99-4501-a815-8cf75872c95c-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.190455 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.190400    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7ctc\" (UniqueName: \"kubernetes.io/projected/124e3837-d436-4036-8940-007ce3e88ab5-kube-api-access-v7ctc\") pod \"node-exporter-pl5jb\" (UID: \"124e3837-d436-4036-8940-007ce3e88ab5\") " pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.191930 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.191903    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg7qj\" (UniqueName: \"kubernetes.io/projected/517464c7-0e99-4501-a815-8cf75872c95c-kube-api-access-lg7qj\") pod \"kube-state-metrics-56d9f944b4-wxzsk\" (UID: \"517464c7-0e99-4501-a815-8cf75872c95c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.259234 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.259194    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"
Jul 28 20:09:52.266424 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.266391    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pl5jb"
Jul 28 20:09:52.582380 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.582340    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95abda6d-c7b6-4b65-87f6-f5a3af15f232-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:52.582589 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:52.582475    2558 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 28 20:09:52.582589 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:52.582549    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95abda6d-c7b6-4b65-87f6-f5a3af15f232-openshift-state-metrics-tls podName:95abda6d-c7b6-4b65-87f6-f5a3af15f232 nodeName:}" failed. No retries permitted until 2026-07-28 20:09:53.582527929 +0000 UTC m=+70.824621134 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/95abda6d-c7b6-4b65-87f6-f5a3af15f232-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-s5gr8" (UID: "95abda6d-c7b6-4b65-87f6-f5a3af15f232") : secret "openshift-state-metrics-tls" not found
Jul 28 20:09:52.880882 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.880766    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 20:09:52.883869 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.883830    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:52.890793 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.890757    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 20:09:52.890973 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.890935    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 20:09:52.891688 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.891632    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 20:09:52.891688 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.891666    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 20:09:52.891854 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.891667    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 20:09:52.892374 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.892088    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 20:09:52.892374 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.892102    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 20:09:52.892374 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.892169    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pkkml\""
Jul 28 20:09:52.892595 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.892386    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 20:09:52.896444 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.896422    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 20:09:52.908043 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.907936    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 20:09:52.985488 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.985439    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:52.985699 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.985550    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:52.985699 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.985613    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-web-config\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:52.985699 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.985644    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwglg\" (UniqueName: \"kubernetes.io/projected/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-kube-api-access-hwglg\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:52.985699 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.985675    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:52.985904 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.985716    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:52.985904 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.985763    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:52.985904 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.985866    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:52.985904 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.985896    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:52.986072 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.985930    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-config-volume\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:52.986072 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.985949    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:52.986072 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.985974    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:52.986072 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:52.985999    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-config-out\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087273 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087229    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087472 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087319    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087472 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087359    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087472 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087396    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-config-volume\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087472 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:53.087444    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-alertmanager-trusted-ca-bundle podName:6fc43d4a-cdec-432d-b0a6-131f8dd2642c nodeName:}" failed. No retries permitted until 2026-07-28 20:09:53.587417132 +0000 UTC m=+70.829510347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "6fc43d4a-cdec-432d-b0a6-131f8dd2642c") : configmap references non-existent config key: ca-bundle.crt
Jul 28 20:09:53.087734 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087576    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087734 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087611    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087734 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087638    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-config-out\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087734 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087678    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087734 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087978 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087755    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-web-config\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087978 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087777    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwglg\" (UniqueName: \"kubernetes.io/projected/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-kube-api-access-hwglg\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087978 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087802    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087978 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087825    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.087978 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.087860    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.088398 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.088373    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.090733 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.090707    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-config-out\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.090891 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.090820    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.091746 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.091695    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-web-config\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.092468 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.092423    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.092721 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.092676    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-config-volume\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.092807 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.092795    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.093012 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.092962    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.093767 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.093738    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.094074 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.094052    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.098599 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.098575    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwglg\" (UniqueName: \"kubernetes.io/projected/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-kube-api-access-hwglg\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.593296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.593251    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.593475 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.593322    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95abda6d-c7b6-4b65-87f6-f5a3af15f232-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:53.594241 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.594207    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fc43d4a-cdec-432d-b0a6-131f8dd2642c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6fc43d4a-cdec-432d-b0a6-131f8dd2642c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:53.596459 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.596432    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95abda6d-c7b6-4b65-87f6-f5a3af15f232-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-s5gr8\" (UID: \"95abda6d-c7b6-4b65-87f6-f5a3af15f232\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:53.717715 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.717674    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"
Jul 28 20:09:53.795370 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:53.795336    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 20:09:54.017962 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:54.017828    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod124e3837_d436_4036_8940_007ce3e88ab5.slice/crio-02ff76098810183f4658f8fa2ae814519e4b53a028b162912fcb152854acb772 WatchSource:0}: Error finding container 02ff76098810183f4658f8fa2ae814519e4b53a028b162912fcb152854acb772: Status 404 returned error can't find the container with id 02ff76098810183f4658f8fa2ae814519e4b53a028b162912fcb152854acb772
Jul 28 20:09:54.225358 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.225317    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk"]
Jul 28 20:09:54.226760 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.226732    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 20:09:54.229975 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.229899    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8"]
Jul 28 20:09:54.230357 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:54.230323    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fc43d4a_cdec_432d_b0a6_131f8dd2642c.slice/crio-0dd38325e231fab671c5d78e5af53583bac6826735cacaaecaeb839a01b81dd4 WatchSource:0}: Error finding container 0dd38325e231fab671c5d78e5af53583bac6826735cacaaecaeb839a01b81dd4: Status 404 returned error can't find the container with id 0dd38325e231fab671c5d78e5af53583bac6826735cacaaecaeb839a01b81dd4
Jul 28 20:09:54.236014 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:54.235871    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod517464c7_0e99_4501_a815_8cf75872c95c.slice/crio-7977e66d6da2e658623969eb116366784e58956b729dfeed42591fa0f9239005 WatchSource:0}: Error finding container 7977e66d6da2e658623969eb116366784e58956b729dfeed42591fa0f9239005: Status 404 returned error can't find the container with id 7977e66d6da2e658623969eb116366784e58956b729dfeed42591fa0f9239005
Jul 28 20:09:54.237498 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:54.237010    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95abda6d_c7b6_4b65_87f6_f5a3af15f232.slice/crio-80f06fd2cd9dc751b378f15f8d97b4250774a6538c107bd1ad27a5bd68700106 WatchSource:0}: Error finding container 80f06fd2cd9dc751b378f15f8d97b4250774a6538c107bd1ad27a5bd68700106: Status 404 returned error can't find the container with id 80f06fd2cd9dc751b378f15f8d97b4250774a6538c107bd1ad27a5bd68700106
Jul 28 20:09:54.790853 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.790815    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8" event={"ID":"95abda6d-c7b6-4b65-87f6-f5a3af15f232","Type":"ContainerStarted","Data":"d1023dbd669f868f301d18c0112d11769403ab37436deaa8602ea5298fd2d7fb"}
Jul 28 20:09:54.791159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.790889    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8" event={"ID":"95abda6d-c7b6-4b65-87f6-f5a3af15f232","Type":"ContainerStarted","Data":"e1f7426febfadbada4b73ca00bdbab80afec603d93a593c42385b27beae0cc1d"}
Jul 28 20:09:54.791159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.790913    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8" event={"ID":"95abda6d-c7b6-4b65-87f6-f5a3af15f232","Type":"ContainerStarted","Data":"80f06fd2cd9dc751b378f15f8d97b4250774a6538c107bd1ad27a5bd68700106"}
Jul 28 20:09:54.793050 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.792955    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q9m9r" event={"ID":"8db6484b-7821-440f-b467-4aa1721dbd50","Type":"ContainerStarted","Data":"738ee91e9dc2098d15b09b7c3f64c8a23c3e0affa10819c8a59d8e68ddccbd45"}
Jul 28 20:09:54.793185 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.793085    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:09:54.795162 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.795062    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zmkbq" event={"ID":"e1b683c6-ec1a-4f37-b354-3a366d2bd899","Type":"ContainerStarted","Data":"f77d2c1640883a57f04b2ab179d3bf37d6f5605b5d1ce01665dc5dc247ef47a4"}
Jul 28 20:09:54.797074 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.796942    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk" event={"ID":"517464c7-0e99-4501-a815-8cf75872c95c","Type":"ContainerStarted","Data":"7977e66d6da2e658623969eb116366784e58956b729dfeed42591fa0f9239005"}
Jul 28 20:09:54.798494 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.798469    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fc43d4a-cdec-432d-b0a6-131f8dd2642c","Type":"ContainerStarted","Data":"0dd38325e231fab671c5d78e5af53583bac6826735cacaaecaeb839a01b81dd4"}
Jul 28 20:09:54.800465 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.800414    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pl5jb" event={"ID":"124e3837-d436-4036-8940-007ce3e88ab5","Type":"ContainerStarted","Data":"02ff76098810183f4658f8fa2ae814519e4b53a028b162912fcb152854acb772"}
Jul 28 20:09:54.814308 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.814246    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q9m9r" podStartSLOduration=66.315608482 podStartE2EDuration="1m10.814222052s" totalImagesPullingTime="4.49861357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:54.813226837 +0000 UTC m=+72.055320061" watchObservedRunningTime="2026-07-28 20:09:54.814222052 +0000 UTC m=+72.056315276"
Jul 28 20:09:54.831508 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.831442    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zmkbq" podStartSLOduration=66.029137032 podStartE2EDuration="1m10.831425307s" totalImagesPullingTime="4.802288275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:54.830942786 +0000 UTC m=+72.073036013" watchObservedRunningTime="2026-07-28 20:09:54.831425307 +0000 UTC m=+72.073518531"
Jul 28 20:09:54.859327 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.858784    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"]
Jul 28 20:09:54.863366 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.863334    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:54.868253 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.868227    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 20:09:54.870583 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.868465    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-55p7r\""
Jul 28 20:09:54.870583 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.868699    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 20:09:54.870583 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.868987    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 20:09:54.870583 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.869174    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 20:09:54.870583 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.869346    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ftpms0ts4dpaj\""
Jul 28 20:09:54.870583 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.870395    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 20:09:54.888371 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.885444    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"]
Jul 28 20:09:54.907968 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.907844    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-grpc-tls\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:54.907968 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.907904    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:54.907968 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.907969    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:54.908274 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.908003    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:54.908274 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.908033    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-tls\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:54.908274 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.908072    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-724qn\" (UniqueName: \"kubernetes.io/projected/0f41c4be-2757-4d01-8117-3a6331c4d413-kube-api-access-724qn\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:54.908274 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.908160    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f41c4be-2757-4d01-8117-3a6331c4d413-metrics-client-ca\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:54.908274 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:54.908189    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.009463 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.009357    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-grpc-tls\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.009463 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.009415    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.010227 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.009786    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.010227 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.009834    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.010227 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.009870    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-tls\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.010227 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.009914    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-724qn\" (UniqueName: \"kubernetes.io/projected/0f41c4be-2757-4d01-8117-3a6331c4d413-kube-api-access-724qn\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.010227 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.009979    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f41c4be-2757-4d01-8117-3a6331c4d413-metrics-client-ca\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.010227 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.010009    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.011198 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.011129    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f41c4be-2757-4d01-8117-3a6331c4d413-metrics-client-ca\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.013103 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.013013    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-tls\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.014010 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.013659    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.014010 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.013846    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.014010 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.013959    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.015263 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.015243    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.015481 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.015460    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0f41c4be-2757-4d01-8117-3a6331c4d413-secret-grpc-tls\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.018928 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.018908    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-724qn\" (UniqueName: \"kubernetes.io/projected/0f41c4be-2757-4d01-8117-3a6331c4d413-kube-api-access-724qn\") pod \"thanos-querier-689b76fbb5-9jsq5\" (UID: \"0f41c4be-2757-4d01-8117-3a6331c4d413\") " pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:55.186359 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:55.186192    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:09:56.030197 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.030088    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"]
Jul 28 20:09:56.052923 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:56.052875    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f41c4be_2757_4d01_8117_3a6331c4d413.slice/crio-188c2a0dba5a5feaa0f6256e0083973af96e0c5c05d8a6ea8d07ca7c0a79a81c WatchSource:0}: Error finding container 188c2a0dba5a5feaa0f6256e0083973af96e0c5c05d8a6ea8d07ca7c0a79a81c: Status 404 returned error can't find the container with id 188c2a0dba5a5feaa0f6256e0083973af96e0c5c05d8a6ea8d07ca7c0a79a81c
Jul 28 20:09:56.369591 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.369540    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-67bf6bb647-f58q9"]
Jul 28 20:09:56.372849 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.372816    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.375569 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.375531    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 20:09:56.375698 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.375637    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 20:09:56.375698 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.375644    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5spmj\""
Jul 28 20:09:56.375788 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.375702    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 20:09:56.376800 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.376779    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-42bihse7tuk1f\""
Jul 28 20:09:56.376925 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.376845    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 20:09:56.383529 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.383505    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67bf6bb647-f58q9"]
Jul 28 20:09:56.423097 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.422996    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c37412a1-56e7-4d17-aa4d-53d08090b1d0-audit-log\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.423097 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.423052    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt6vj\" (UniqueName: \"kubernetes.io/projected/c37412a1-56e7-4d17-aa4d-53d08090b1d0-kube-api-access-tt6vj\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.423097 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.423082    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c37412a1-56e7-4d17-aa4d-53d08090b1d0-metrics-server-audit-profiles\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.423336 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.423140    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c37412a1-56e7-4d17-aa4d-53d08090b1d0-secret-metrics-server-tls\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.423336 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.423170    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c37412a1-56e7-4d17-aa4d-53d08090b1d0-secret-metrics-server-client-certs\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.423336 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.423216    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c37412a1-56e7-4d17-aa4d-53d08090b1d0-client-ca-bundle\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.423336 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.423290    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c37412a1-56e7-4d17-aa4d-53d08090b1d0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.524150 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.524110    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c37412a1-56e7-4d17-aa4d-53d08090b1d0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.524330 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.524180    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c37412a1-56e7-4d17-aa4d-53d08090b1d0-audit-log\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.524330 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.524222    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt6vj\" (UniqueName: \"kubernetes.io/projected/c37412a1-56e7-4d17-aa4d-53d08090b1d0-kube-api-access-tt6vj\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.524330 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.524248    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c37412a1-56e7-4d17-aa4d-53d08090b1d0-metrics-server-audit-profiles\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.524330 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.524288    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c37412a1-56e7-4d17-aa4d-53d08090b1d0-secret-metrics-server-tls\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.524330 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.524318    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c37412a1-56e7-4d17-aa4d-53d08090b1d0-secret-metrics-server-client-certs\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.524603 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.524345    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c37412a1-56e7-4d17-aa4d-53d08090b1d0-client-ca-bundle\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.524754 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.524731    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c37412a1-56e7-4d17-aa4d-53d08090b1d0-audit-log\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.525074 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.525045    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c37412a1-56e7-4d17-aa4d-53d08090b1d0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.525278 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.525256    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c37412a1-56e7-4d17-aa4d-53d08090b1d0-metrics-server-audit-profiles\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.527296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.527273    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c37412a1-56e7-4d17-aa4d-53d08090b1d0-secret-metrics-server-client-certs\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.527396 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.527353    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c37412a1-56e7-4d17-aa4d-53d08090b1d0-secret-metrics-server-tls\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.527576 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.527469    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"]
Jul 28 20:09:56.527890 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.527873    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c37412a1-56e7-4d17-aa4d-53d08090b1d0-client-ca-bundle\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.532369 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.532349    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"
Jul 28 20:09:56.534975 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.534952    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt6vj\" (UniqueName: \"kubernetes.io/projected/c37412a1-56e7-4d17-aa4d-53d08090b1d0-kube-api-access-tt6vj\") pod \"metrics-server-67bf6bb647-f58q9\" (UID: \"c37412a1-56e7-4d17-aa4d-53d08090b1d0\") " pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.535284 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.535265    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 20:09:56.535369 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.535312    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6tclw\""
Jul 28 20:09:56.539050 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.539028    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f46b6dcfb-8p2mw"]
Jul 28 20:09:56.543126 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.543097    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"]
Jul 28 20:09:56.574673 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.574643    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84685cbd4f-qwh9w"]
Jul 28 20:09:56.578245 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.578217    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.588522 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.588494    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84685cbd4f-qwh9w"]
Jul 28 20:09:56.625839 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.625799    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-oauth-serving-cert\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.625993 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.625853    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppkxs\" (UniqueName: \"kubernetes.io/projected/26107ef4-9268-4af8-aeba-72115376a2ae-kube-api-access-ppkxs\") pod \"monitoring-plugin-68b7fc9574-tw4j4\" (UID: \"26107ef4-9268-4af8-aeba-72115376a2ae\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"
Jul 28 20:09:56.625993 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.625949    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-console-config\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.626170 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.625994    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-service-ca\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.626170 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.626016    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41f107b2-411f-4631-a6f0-59ef867edc19-console-serving-cert\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.626170 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.626064    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41f107b2-411f-4631-a6f0-59ef867edc19-console-oauth-config\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.626170 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.626088    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26107ef4-9268-4af8-aeba-72115376a2ae-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-tw4j4\" (UID: \"26107ef4-9268-4af8-aeba-72115376a2ae\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"
Jul 28 20:09:56.626170 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.626137    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfg7r\" (UniqueName: \"kubernetes.io/projected/41f107b2-411f-4631-a6f0-59ef867edc19-kube-api-access-vfg7r\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.626170 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.626166    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-trusted-ca-bundle\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.682697 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.682606    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:09:56.727286 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.727246    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppkxs\" (UniqueName: \"kubernetes.io/projected/26107ef4-9268-4af8-aeba-72115376a2ae-kube-api-access-ppkxs\") pod \"monitoring-plugin-68b7fc9574-tw4j4\" (UID: \"26107ef4-9268-4af8-aeba-72115376a2ae\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"
Jul 28 20:09:56.727469 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.727299    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-console-config\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.727469 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.727324    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-service-ca\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.727469 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.727346    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41f107b2-411f-4631-a6f0-59ef867edc19-console-serving-cert\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.727469 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.727368    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41f107b2-411f-4631-a6f0-59ef867edc19-console-oauth-config\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.727469 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.727394    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26107ef4-9268-4af8-aeba-72115376a2ae-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-tw4j4\" (UID: \"26107ef4-9268-4af8-aeba-72115376a2ae\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"
Jul 28 20:09:56.727469 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.727444    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfg7r\" (UniqueName: \"kubernetes.io/projected/41f107b2-411f-4631-a6f0-59ef867edc19-kube-api-access-vfg7r\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.727796 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.727473    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-trusted-ca-bundle\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.727796 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.727516    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-oauth-serving-cert\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.727796 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:56.727650    2558 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 28 20:09:56.727796 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:09:56.727723    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26107ef4-9268-4af8-aeba-72115376a2ae-monitoring-plugin-cert podName:26107ef4-9268-4af8-aeba-72115376a2ae nodeName:}" failed. No retries permitted until 2026-07-28 20:09:57.227701401 +0000 UTC m=+74.469794606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/26107ef4-9268-4af8-aeba-72115376a2ae-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-tw4j4" (UID: "26107ef4-9268-4af8-aeba-72115376a2ae") : secret "monitoring-plugin-cert" not found
Jul 28 20:09:56.728201 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.728165    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-service-ca\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.728369 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.728309    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-oauth-serving-cert\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.728434 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.728368    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-console-config\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.728594 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.728547    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-trusted-ca-bundle\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.730096 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.730057    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41f107b2-411f-4631-a6f0-59ef867edc19-console-serving-cert\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.730196 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.730175    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41f107b2-411f-4631-a6f0-59ef867edc19-console-oauth-config\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.743047 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.743009    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppkxs\" (UniqueName: \"kubernetes.io/projected/26107ef4-9268-4af8-aeba-72115376a2ae-kube-api-access-ppkxs\") pod \"monitoring-plugin-68b7fc9574-tw4j4\" (UID: \"26107ef4-9268-4af8-aeba-72115376a2ae\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"
Jul 28 20:09:56.743170 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.743065    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfg7r\" (UniqueName: \"kubernetes.io/projected/41f107b2-411f-4631-a6f0-59ef867edc19-kube-api-access-vfg7r\") pod \"console-84685cbd4f-qwh9w\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") " pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.810851 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.810721    2558 generic.go:356] "Generic (PLEG): container finished" podID="6fc43d4a-cdec-432d-b0a6-131f8dd2642c" containerID="2551d3e5c9b65a458fd9512303360bbcbd88bb2d460d77d481aa66a0d88be12a" exitCode=0
Jul 28 20:09:56.810851 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.810770    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fc43d4a-cdec-432d-b0a6-131f8dd2642c","Type":"ContainerDied","Data":"2551d3e5c9b65a458fd9512303360bbcbd88bb2d460d77d481aa66a0d88be12a"}
Jul 28 20:09:56.813264 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.813235    2558 generic.go:356] "Generic (PLEG): container finished" podID="124e3837-d436-4036-8940-007ce3e88ab5" containerID="46089b5ce238b3099a4abdb3d49593b311104d314578206433b8d409814446e5" exitCode=0
Jul 28 20:09:56.813362 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.813332    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pl5jb" event={"ID":"124e3837-d436-4036-8940-007ce3e88ab5","Type":"ContainerDied","Data":"46089b5ce238b3099a4abdb3d49593b311104d314578206433b8d409814446e5"}
Jul 28 20:09:56.815947 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.815917    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5" event={"ID":"0f41c4be-2757-4d01-8117-3a6331c4d413","Type":"ContainerStarted","Data":"188c2a0dba5a5feaa0f6256e0083973af96e0c5c05d8a6ea8d07ca7c0a79a81c"}
Jul 28 20:09:56.821156 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.821125    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8" event={"ID":"95abda6d-c7b6-4b65-87f6-f5a3af15f232","Type":"ContainerStarted","Data":"b6ea90b5d9d1cb373799eff89cf4961e448e3f7c64fbbc25e090791345439f6d"}
Jul 28 20:09:56.823828 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.823801    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk" event={"ID":"517464c7-0e99-4501-a815-8cf75872c95c","Type":"ContainerStarted","Data":"9a01f47310e75010105c414471ae367be19884e0da5d24ef7e88b34c60e48495"}
Jul 28 20:09:56.823941 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.823864    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk" event={"ID":"517464c7-0e99-4501-a815-8cf75872c95c","Type":"ContainerStarted","Data":"c1d7f56725d23ec0747917bfaab9719d99e1c18711197eaa6a8bec385e5600b4"}
Jul 28 20:09:56.823941 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.823896    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk" event={"ID":"517464c7-0e99-4501-a815-8cf75872c95c","Type":"ContainerStarted","Data":"bb751a6aee7ef23fcc2c564c791b5ccbb158c2ed5161915541057ec4e1054fda"}
Jul 28 20:09:56.832722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.832692    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67bf6bb647-f58q9"]
Jul 28 20:09:56.838454 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:56.838398    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc37412a1_56e7_4d17_aa4d_53d08090b1d0.slice/crio-1c360fabf471616210e71b38078dc2ab30d35d7885f3f5565d1eacd9c30f9563 WatchSource:0}: Error finding container 1c360fabf471616210e71b38078dc2ab30d35d7885f3f5565d1eacd9c30f9563: Status 404 returned error can't find the container with id 1c360fabf471616210e71b38078dc2ab30d35d7885f3f5565d1eacd9c30f9563
Jul 28 20:09:56.861884 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.861199    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s5gr8" podStartSLOduration=4.338661978 podStartE2EDuration="5.861177878s" totalImagesPullingTime="1.5225159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:56.86104396 +0000 UTC m=+74.103137182" watchObservedRunningTime="2026-07-28 20:09:56.861177878 +0000 UTC m=+74.103271102"
Jul 28 20:09:56.890293 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.890248    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:09:56.900286 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:56.900169    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-wxzsk" podStartSLOduration=4.272973175 podStartE2EDuration="5.900146276s" totalImagesPullingTime="1.627173101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:56.899529602 +0000 UTC m=+74.141622826" watchObservedRunningTime="2026-07-28 20:09:56.900146276 +0000 UTC m=+74.142239500"
Jul 28 20:09:57.044019 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.043984    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84685cbd4f-qwh9w"]
Jul 28 20:09:57.047213 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:57.047181    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41f107b2_411f_4631_a6f0_59ef867edc19.slice/crio-e52d48c952e9a78f1bd72fed75e0d0a7c2ff8dcb0eb991945715bdfb0c88836b WatchSource:0}: Error finding container e52d48c952e9a78f1bd72fed75e0d0a7c2ff8dcb0eb991945715bdfb0c88836b: Status 404 returned error can't find the container with id e52d48c952e9a78f1bd72fed75e0d0a7c2ff8dcb0eb991945715bdfb0c88836b
Jul 28 20:09:57.163715 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.163680    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"]
Jul 28 20:09:57.167783 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.167757    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.170683 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.170657    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 20:09:57.170847 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.170657    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 20:09:57.172152 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.172127    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 20:09:57.172535 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.172513    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 20:09:57.172678 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.172534    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 20:09:57.173258 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.173238    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-9pv96\""
Jul 28 20:09:57.177408 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.177374    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 20:09:57.188854 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.188715    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"]
Jul 28 20:09:57.231982 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.231930    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzrbz\" (UniqueName: \"kubernetes.io/projected/778eeadb-b700-411e-9367-8112cd14d750-kube-api-access-jzrbz\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.231982 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.231983    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/778eeadb-b700-411e-9367-8112cd14d750-telemeter-client-tls\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.232191 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.232036    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/778eeadb-b700-411e-9367-8112cd14d750-serving-certs-ca-bundle\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.232191 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.232064    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/778eeadb-b700-411e-9367-8112cd14d750-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.232191 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.232105    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/778eeadb-b700-411e-9367-8112cd14d750-federate-client-tls\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.232191 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.232149    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26107ef4-9268-4af8-aeba-72115376a2ae-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-tw4j4\" (UID: \"26107ef4-9268-4af8-aeba-72115376a2ae\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"
Jul 28 20:09:57.232191 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.232185    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/778eeadb-b700-411e-9367-8112cd14d750-secret-telemeter-client\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.232441 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.232211    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/778eeadb-b700-411e-9367-8112cd14d750-metrics-client-ca\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.232441 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.232236    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/778eeadb-b700-411e-9367-8112cd14d750-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.235423 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.235394    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26107ef4-9268-4af8-aeba-72115376a2ae-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-tw4j4\" (UID: \"26107ef4-9268-4af8-aeba-72115376a2ae\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"
Jul 28 20:09:57.333642 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.333597    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzrbz\" (UniqueName: \"kubernetes.io/projected/778eeadb-b700-411e-9367-8112cd14d750-kube-api-access-jzrbz\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.333642 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.333643    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/778eeadb-b700-411e-9367-8112cd14d750-telemeter-client-tls\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.333890 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.333686    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/778eeadb-b700-411e-9367-8112cd14d750-serving-certs-ca-bundle\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.333890 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.333724    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/778eeadb-b700-411e-9367-8112cd14d750-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.335969 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.334051    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/778eeadb-b700-411e-9367-8112cd14d750-federate-client-tls\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.335969 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.334121    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/778eeadb-b700-411e-9367-8112cd14d750-secret-telemeter-client\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.335969 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.334148    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/778eeadb-b700-411e-9367-8112cd14d750-metrics-client-ca\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.335969 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.334173    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/778eeadb-b700-411e-9367-8112cd14d750-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.335969 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.334636    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/778eeadb-b700-411e-9367-8112cd14d750-serving-certs-ca-bundle\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.335969 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.335104    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/778eeadb-b700-411e-9367-8112cd14d750-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.335969 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.335273    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/778eeadb-b700-411e-9367-8112cd14d750-metrics-client-ca\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.337049 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.337023    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/778eeadb-b700-411e-9367-8112cd14d750-telemeter-client-tls\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.337811 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.337658    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/778eeadb-b700-411e-9367-8112cd14d750-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.337811 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.337770    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/778eeadb-b700-411e-9367-8112cd14d750-federate-client-tls\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.338281 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.338258    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/778eeadb-b700-411e-9367-8112cd14d750-secret-telemeter-client\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.344309 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.344194    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzrbz\" (UniqueName: \"kubernetes.io/projected/778eeadb-b700-411e-9367-8112cd14d750-kube-api-access-jzrbz\") pod \"telemeter-client-5b65dfccdf-vvwkd\" (UID: \"778eeadb-b700-411e-9367-8112cd14d750\") " pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.457127 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.457038    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"
Jul 28 20:09:57.480720 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.480686    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"
Jul 28 20:09:57.630055 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.630023    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"]
Jul 28 20:09:57.633395 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:57.633363    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26107ef4_9268_4af8_aeba_72115376a2ae.slice/crio-6adb40b9787476e9768289a926c4babc458ed2f470ca59005ae4233993dfa64b WatchSource:0}: Error finding container 6adb40b9787476e9768289a926c4babc458ed2f470ca59005ae4233993dfa64b: Status 404 returned error can't find the container with id 6adb40b9787476e9768289a926c4babc458ed2f470ca59005ae4233993dfa64b
Jul 28 20:09:57.660581 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.660532    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd"]
Jul 28 20:09:57.836008 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.835897    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pl5jb" event={"ID":"124e3837-d436-4036-8940-007ce3e88ab5","Type":"ContainerStarted","Data":"92d2f08ba52862aa372aa766de0e0c95bd56ffec4d3b6fd761866c09cb6142ed"}
Jul 28 20:09:57.836008 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.835970    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pl5jb" event={"ID":"124e3837-d436-4036-8940-007ce3e88ab5","Type":"ContainerStarted","Data":"7eeb5665bcd466289649ed40da46020ae0df59956e39fe239d96a07a1715b400"}
Jul 28 20:09:57.838948 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.838909    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84685cbd4f-qwh9w" event={"ID":"41f107b2-411f-4631-a6f0-59ef867edc19","Type":"ContainerStarted","Data":"3f4718b3858062b9abd85b2fcefc22e521aabaadf65fecf3c6e46871b1997571"}
Jul 28 20:09:57.839101 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.838967    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84685cbd4f-qwh9w" event={"ID":"41f107b2-411f-4631-a6f0-59ef867edc19","Type":"ContainerStarted","Data":"e52d48c952e9a78f1bd72fed75e0d0a7c2ff8dcb0eb991945715bdfb0c88836b"}
Jul 28 20:09:57.841028 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.840957    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9" event={"ID":"c37412a1-56e7-4d17-aa4d-53d08090b1d0","Type":"ContainerStarted","Data":"1c360fabf471616210e71b38078dc2ab30d35d7885f3f5565d1eacd9c30f9563"}
Jul 28 20:09:57.842701 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.842670    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4" event={"ID":"26107ef4-9268-4af8-aeba-72115376a2ae","Type":"ContainerStarted","Data":"6adb40b9787476e9768289a926c4babc458ed2f470ca59005ae4233993dfa64b"}
Jul 28 20:09:57.859582 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.859481    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pl5jb" podStartSLOduration=4.921486468 podStartE2EDuration="6.85945899s" totalImagesPullingTime="1.836422387s" totalInitContainerRuntime="101.550135ms" isStatefulPod=true podCreationTimestamp="2026-07-28 20:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:57.858767332 +0000 UTC m=+75.100860558" watchObservedRunningTime="2026-07-28 20:09:57.85945899 +0000 UTC m=+75.101552214"
Jul 28 20:09:57.880093 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:57.880038    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84685cbd4f-qwh9w" podStartSLOduration=1.880023323 podStartE2EDuration="1.880023323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:57.879897291 +0000 UTC m=+75.121990514" watchObservedRunningTime="2026-07-28 20:09:57.880023323 +0000 UTC m=+75.122116547"
Jul 28 20:09:58.096423 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.096200    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 20:09:58.100983 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.100951    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.104612 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.104061    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 20:09:58.104612 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.104314    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 20:09:58.104612 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.104493    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 20:09:58.105000 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.104979    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 20:09:58.105105 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.105059    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 20:09:58.105105 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.105069    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 20:09:58.105212 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.105114    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 20:09:58.105212 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.105059    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 20:09:58.105272 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.105214    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rxh4d\""
Jul 28 20:09:58.105378 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.105356    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 20:09:58.105481 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.105405    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 20:09:58.105791 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.105607    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 20:09:58.105791 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.105631    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 20:09:58.105791 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.105671    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-brmd90v13ae6i\""
Jul 28 20:09:58.105791 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.105678    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 20:09:58.107011 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.106994    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 20:09:58.122081 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.122034    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245386    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245438    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245461    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245488    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245512    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245536    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245582    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245611    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-web-config\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245648    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-config\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245672    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245700    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wlk7\" (UniqueName: \"kubernetes.io/projected/921afb17-7890-44c5-81f9-01736a22bfae-kube-api-access-4wlk7\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245732    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245763    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/921afb17-7890-44c5-81f9-01736a22bfae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245786    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245824    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/921afb17-7890-44c5-81f9-01736a22bfae-config-out\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.246387 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245850    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.247317 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.245901    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.247317 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.246179    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.247317 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.246208    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.247317 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.246245    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.295262 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:58.295221    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod778eeadb_b700_411e_9367_8112cd14d750.slice/crio-0cfaea2f2104761b28ebfaf801c236fa55edeeceb290f0f50682194e20e8955e WatchSource:0}: Error finding container 0cfaea2f2104761b28ebfaf801c236fa55edeeceb290f0f50682194e20e8955e: Status 404 returned error can't find the container with id 0cfaea2f2104761b28ebfaf801c236fa55edeeceb290f0f50682194e20e8955e
Jul 28 20:09:58.347775 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.347673    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.347775 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.347756    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348005 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.347793    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348005 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.347815    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348005 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.347840    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348005 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.347864    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348005 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.347887    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348005 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.347908    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348005 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.347934    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-web-config\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348005 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.347969    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-config\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348005 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.347995    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348394 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.348025    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wlk7\" (UniqueName: \"kubernetes.io/projected/921afb17-7890-44c5-81f9-01736a22bfae-kube-api-access-4wlk7\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348394 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.348055    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348394 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.348082    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/921afb17-7890-44c5-81f9-01736a22bfae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348394 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.348105    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348394 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.348141    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/921afb17-7890-44c5-81f9-01736a22bfae-config-out\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348394 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.348167    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348394 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.348213    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348394 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.348246    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348394 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.348270    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.348813 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.348520    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.350773 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.349499    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.350773 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.350056    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.350773 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.350421    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.352546 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.352211    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.352691 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.352647    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.353424 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.353363    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.355779 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.355728    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.356785 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.356714    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/921afb17-7890-44c5-81f9-01736a22bfae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.356785 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.356744    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.357297 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.357267    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.357474 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.357384    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.357967 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.357921    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/921afb17-7890-44c5-81f9-01736a22bfae-config-out\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.358065 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.357977    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/921afb17-7890-44c5-81f9-01736a22bfae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.358243 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.358225    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.358320 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.358284    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.358916 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.358893    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wlk7\" (UniqueName: \"kubernetes.io/projected/921afb17-7890-44c5-81f9-01736a22bfae-kube-api-access-4wlk7\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.359010 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.358898    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-config\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.359169 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.359149    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.360041 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.360021    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/921afb17-7890-44c5-81f9-01736a22bfae-web-config\") pod \"prometheus-k8s-0\" (UID: \"921afb17-7890-44c5-81f9-01736a22bfae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.428166 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.428127    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:09:58.852220 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.851862    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fc43d4a-cdec-432d-b0a6-131f8dd2642c","Type":"ContainerStarted","Data":"82cc1aac26be1ffffd8e659fcf79c9b44c401f2bd3e632982e099a1a6af6e709"}
Jul 28 20:09:58.856305 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.856215    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5" event={"ID":"0f41c4be-2757-4d01-8117-3a6331c4d413","Type":"ContainerStarted","Data":"0c91f021f1b67826d6f27d74fdb37771c0728d716d32bb43ee5bd4ea7e3b78d7"}
Jul 28 20:09:58.858959 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.858616    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd" event={"ID":"778eeadb-b700-411e-9367-8112cd14d750","Type":"ContainerStarted","Data":"0cfaea2f2104761b28ebfaf801c236fa55edeeceb290f0f50682194e20e8955e"}
Jul 28 20:09:58.928262 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:58.928169    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 20:09:58.930801 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:09:58.930764    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod921afb17_7890_44c5_81f9_01736a22bfae.slice/crio-5f333f66812e156ab876d7e5c788d1185bac7804202c298c056dd93eef669051 WatchSource:0}: Error finding container 5f333f66812e156ab876d7e5c788d1185bac7804202c298c056dd93eef669051: Status 404 returned error can't find the container with id 5f333f66812e156ab876d7e5c788d1185bac7804202c298c056dd93eef669051
Jul 28 20:09:59.864587 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:59.864348    2558 generic.go:356] "Generic (PLEG): container finished" podID="921afb17-7890-44c5-81f9-01736a22bfae" containerID="7f34be48ad52e9c472588c5351acd721863da0a23dd555c820ff2968740b5e54" exitCode=0
Jul 28 20:09:59.864587 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:59.864455    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"921afb17-7890-44c5-81f9-01736a22bfae","Type":"ContainerDied","Data":"7f34be48ad52e9c472588c5351acd721863da0a23dd555c820ff2968740b5e54"}
Jul 28 20:09:59.864587 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:59.864550    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"921afb17-7890-44c5-81f9-01736a22bfae","Type":"ContainerStarted","Data":"5f333f66812e156ab876d7e5c788d1185bac7804202c298c056dd93eef669051"}
Jul 28 20:09:59.871706 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:59.871678    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fc43d4a-cdec-432d-b0a6-131f8dd2642c","Type":"ContainerStarted","Data":"7073d39f9e7b79fdfe15668b3652cc5a565fde07f389fa7b765296e4ddf59e47"}
Jul 28 20:09:59.871832 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:59.871753    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fc43d4a-cdec-432d-b0a6-131f8dd2642c","Type":"ContainerStarted","Data":"364fdbd5b8422605082b3a46de392c7efa525127c9fc6c02538e284b0c69cff8"}
Jul 28 20:09:59.871832 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:59.871793    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fc43d4a-cdec-432d-b0a6-131f8dd2642c","Type":"ContainerStarted","Data":"e460ae2d52c55e42056ddf7602aa42d14c3db6e2295b3fcb196b9c7997a55a9a"}
Jul 28 20:09:59.871960 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:59.871835    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fc43d4a-cdec-432d-b0a6-131f8dd2642c","Type":"ContainerStarted","Data":"2e7827712a58e40322c0318487ac9f84d90b3c3221d142a0aa836304ff463e41"}
Jul 28 20:09:59.874199 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:59.874176    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5" event={"ID":"0f41c4be-2757-4d01-8117-3a6331c4d413","Type":"ContainerStarted","Data":"4ae052bcea68387644f005541c88a3b04b7c0d51343195c1845361fdedf45fd7"}
Jul 28 20:09:59.874305 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:09:59.874244    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5" event={"ID":"0f41c4be-2757-4d01-8117-3a6331c4d413","Type":"ContainerStarted","Data":"94b7f36abe0bd69a15d315fcb40ff61ffa0247d0358068e2f68e27cee56b8d58"}
Jul 28 20:10:01.886032 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.885992    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4" event={"ID":"26107ef4-9268-4af8-aeba-72115376a2ae","Type":"ContainerStarted","Data":"89ab2426a94e4a276e56bb6d5e1f54262932c171913b968faf00fe9ca9dd09a3"}
Jul 28 20:10:01.886494 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.886190    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"
Jul 28 20:10:01.890519 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.890494    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6fc43d4a-cdec-432d-b0a6-131f8dd2642c","Type":"ContainerStarted","Data":"fcb3ef54390d2cb939370321ee0ffbb8a66faf6d7fd919f8bdfeb53cd3a67411"}
Jul 28 20:10:01.892385 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.892365    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4"
Jul 28 20:10:01.893774 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.893754    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5" event={"ID":"0f41c4be-2757-4d01-8117-3a6331c4d413","Type":"ContainerStarted","Data":"84c575fffe16e603f085a656aa463c9ef4aa3b03f72d14fc078a3b6cb34e418f"}
Jul 28 20:10:01.893867 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.893804    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5" event={"ID":"0f41c4be-2757-4d01-8117-3a6331c4d413","Type":"ContainerStarted","Data":"ed0c4dc1f5ae3800895f94269632947c5a74c1e7714d267635f49b8f8647a022"}
Jul 28 20:10:01.893867 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.893832    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5" event={"ID":"0f41c4be-2757-4d01-8117-3a6331c4d413","Type":"ContainerStarted","Data":"d7f4fe3d3a6584010004ef6e32c51308e6d669b1e26693ab2d248d46e520e645"}
Jul 28 20:10:01.893957 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.893876    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:10:01.895845 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.895823    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd" event={"ID":"778eeadb-b700-411e-9367-8112cd14d750","Type":"ContainerStarted","Data":"e104f5fb5b82ee3cfbf900817e4d80e440f6cb3d01755f65607245935988cd15"}
Jul 28 20:10:01.895937 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.895877    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd" event={"ID":"778eeadb-b700-411e-9367-8112cd14d750","Type":"ContainerStarted","Data":"7a1abdab50fee1ce97b3f26736cfaa3ccd77fe4e176301ecc61c3d56515fa9e1"}
Jul 28 20:10:01.895937 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.895899    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd" event={"ID":"778eeadb-b700-411e-9367-8112cd14d750","Type":"ContainerStarted","Data":"3f9df6fea24cbba78f1fbb33e45b2cedbfe41445845f3eb5ec2a003520dc7011"}
Jul 28 20:10:01.897275 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.897253    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9" event={"ID":"c37412a1-56e7-4d17-aa4d-53d08090b1d0","Type":"ContainerStarted","Data":"c68954715b1d82c47f7dfa33ebdf64d1fdd38987f54b7acd6bc087a8e1421ad7"}
Jul 28 20:10:01.903822 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.903784    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-tw4j4" podStartSLOduration=2.47689084 podStartE2EDuration="5.90376932s" totalImagesPullingTime="3.42687848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:10:01.90292997 +0000 UTC m=+79.145023194" watchObservedRunningTime="2026-07-28 20:10:01.90376932 +0000 UTC m=+79.145862543"
Jul 28 20:10:01.922308 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.922251    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9" podStartSLOduration=1.7008736020000002 podStartE2EDuration="5.922233448s" totalImagesPullingTime="4.221359846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:10:01.920230845 +0000 UTC m=+79.162324062" watchObservedRunningTime="2026-07-28 20:10:01.922233448 +0000 UTC m=+79.164326648"
Jul 28 20:10:01.944091 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.944025    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5" podStartSLOduration=3.183513781 podStartE2EDuration="7.943989082s" totalImagesPullingTime="4.760475301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:10:01.941855889 +0000 UTC m=+79.183949115" watchObservedRunningTime="2026-07-28 20:10:01.943989082 +0000 UTC m=+79.186082305"
Jul 28 20:10:01.966338 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.966282    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5b65dfccdf-vvwkd" podStartSLOduration=2.19582619 podStartE2EDuration="4.966265027s" totalImagesPullingTime="2.770438837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:10:01.965071811 +0000 UTC m=+79.207165035" watchObservedRunningTime="2026-07-28 20:10:01.966265027 +0000 UTC m=+79.208358250"
Jul 28 20:10:01.997029 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:01.996967    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.554978753 podStartE2EDuration="9.99694296s" totalImagesPullingTime="5.415426194s" totalInitContainerRuntime="26.538013ms" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:10:01.994231164 +0000 UTC m=+79.236324389" watchObservedRunningTime="2026-07-28 20:10:01.99694296 +0000 UTC m=+79.239036184"
Jul 28 20:10:02.841720 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:02.841660    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84685cbd4f-qwh9w"]
Jul 28 20:10:04.917496 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:04.917462    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"921afb17-7890-44c5-81f9-01736a22bfae","Type":"ContainerStarted","Data":"5706617c004616fec9e08fd38e98a763da859d78f7378aa44419cb3ea0de80b4"}
Jul 28 20:10:04.917922 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:04.917528    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"921afb17-7890-44c5-81f9-01736a22bfae","Type":"ContainerStarted","Data":"189c83fa86774df507cc4c57a2173a2d459427176a86be0564f994e938a559f7"}
Jul 28 20:10:04.917922 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:04.917553    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"921afb17-7890-44c5-81f9-01736a22bfae","Type":"ContainerStarted","Data":"da188f4c2478f94716f0dab97cc919a3ff3b661fbe98c51ada5d88244517b136"}
Jul 28 20:10:04.917922 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:04.917597    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"921afb17-7890-44c5-81f9-01736a22bfae","Type":"ContainerStarted","Data":"85f85d5816fb0f6201eb6119eebfef7c655ae78477a5fa552101bb13f60f4aeb"}
Jul 28 20:10:04.917922 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:04.917630    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"921afb17-7890-44c5-81f9-01736a22bfae","Type":"ContainerStarted","Data":"5541be452c4905d930e62b18fa67836d7ee37ec57b520ec58a050d11433a106c"}
Jul 28 20:10:04.917922 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:04.917657    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"921afb17-7890-44c5-81f9-01736a22bfae","Type":"ContainerStarted","Data":"b5bf23718fc47a59882d891d600901234827918d95108434a35d8d7e0715b94b"}
Jul 28 20:10:04.955381 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:04.955294    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.558663144 podStartE2EDuration="6.955240418s" totalImagesPullingTime="4.352189497s" totalInitContainerRuntime="44.387777ms" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:10:04.952702495 +0000 UTC m=+82.194795718" watchObservedRunningTime="2026-07-28 20:10:04.955240418 +0000 UTC m=+82.197333643"
Jul 28 20:10:06.890843 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:06.890798    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:10:07.911406 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:07.911380    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-689b76fbb5-9jsq5"
Jul 28 20:10:08.429195 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:08.429154    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:10:16.683172 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:16.683132    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:10:16.683597 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:16.683209    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:10:21.560921 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:21.560852    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f46b6dcfb-8p2mw" podUID="9be6a081-53cb-449d-a9d7-ecfa5a37a683" containerName="console" containerID="cri-o://ae1b7cbd3357f2f1fd5c27e7dde6d4db089d902dbed7482d4f2263b1ac6bc8cd" gracePeriod=15
Jul 28 20:10:21.993710 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:21.993678    2558 generic.go:356] "Generic (PLEG): container finished" podID="9be6a081-53cb-449d-a9d7-ecfa5a37a683" containerID="ae1b7cbd3357f2f1fd5c27e7dde6d4db089d902dbed7482d4f2263b1ac6bc8cd" exitCode=0
Jul 28 20:10:21.993880 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:21.993751    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f46b6dcfb-8p2mw" event={"ID":"9be6a081-53cb-449d-a9d7-ecfa5a37a683","Type":"ContainerDied","Data":"ae1b7cbd3357f2f1fd5c27e7dde6d4db089d902dbed7482d4f2263b1ac6bc8cd"}
Jul 28 20:10:22.336965 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.336926    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:10:22.384431 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.384397    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-service-ca\") pod \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") "
Jul 28 20:10:22.384657 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.384465    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-serving-cert\") pod \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") "
Jul 28 20:10:22.384657 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.384488    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-oauth-config\") pod \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") "
Jul 28 20:10:22.384657 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.384551    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-trusted-ca-bundle\") pod \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") "
Jul 28 20:10:22.384657 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.384615    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-config\") pod \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") "
Jul 28 20:10:22.384657 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.384649    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-oauth-serving-cert\") pod \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") "
Jul 28 20:10:22.384926 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.384812    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-service-ca" (OuterVolumeSpecName: "service-ca") pod "9be6a081-53cb-449d-a9d7-ecfa5a37a683" (UID: "9be6a081-53cb-449d-a9d7-ecfa5a37a683"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:10:22.385092 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.385064    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnlhq\" (UniqueName: \"kubernetes.io/projected/9be6a081-53cb-449d-a9d7-ecfa5a37a683-kube-api-access-pnlhq\") pod \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\" (UID: \"9be6a081-53cb-449d-a9d7-ecfa5a37a683\") "
Jul 28 20:10:22.385092 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.385076    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-config" (OuterVolumeSpecName: "console-config") pod "9be6a081-53cb-449d-a9d7-ecfa5a37a683" (UID: "9be6a081-53cb-449d-a9d7-ecfa5a37a683"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:10:22.385092 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.385082    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9be6a081-53cb-449d-a9d7-ecfa5a37a683" (UID: "9be6a081-53cb-449d-a9d7-ecfa5a37a683"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:10:22.385346 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.385189    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9be6a081-53cb-449d-a9d7-ecfa5a37a683" (UID: "9be6a081-53cb-449d-a9d7-ecfa5a37a683"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:10:22.385405 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.385382    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-service-ca\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:22.385461 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.385404    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-trusted-ca-bundle\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:22.385461 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.385419    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-config\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:22.385461 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.385433    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9be6a081-53cb-449d-a9d7-ecfa5a37a683-oauth-serving-cert\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:22.387096 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.387063    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9be6a081-53cb-449d-a9d7-ecfa5a37a683" (UID: "9be6a081-53cb-449d-a9d7-ecfa5a37a683"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:10:22.387196 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.387108    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9be6a081-53cb-449d-a9d7-ecfa5a37a683-kube-api-access-pnlhq" (OuterVolumeSpecName: "kube-api-access-pnlhq") pod "9be6a081-53cb-449d-a9d7-ecfa5a37a683" (UID: "9be6a081-53cb-449d-a9d7-ecfa5a37a683"). InnerVolumeSpecName "kube-api-access-pnlhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:10:22.387196 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.387115    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9be6a081-53cb-449d-a9d7-ecfa5a37a683" (UID: "9be6a081-53cb-449d-a9d7-ecfa5a37a683"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:10:22.486724 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.486681    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-serving-cert\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:22.486724 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.486716    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9be6a081-53cb-449d-a9d7-ecfa5a37a683-console-oauth-config\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:22.486724 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.486726    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnlhq\" (UniqueName: \"kubernetes.io/projected/9be6a081-53cb-449d-a9d7-ecfa5a37a683-kube-api-access-pnlhq\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:22.999674 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.999635    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f46b6dcfb-8p2mw" event={"ID":"9be6a081-53cb-449d-a9d7-ecfa5a37a683","Type":"ContainerDied","Data":"989e44579e42500937785866bf96574a67e1f5b047a3641984222867cde5429a"}
Jul 28 20:10:23.000044 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.999703    2558 scope.go:117] "RemoveContainer" containerID="ae1b7cbd3357f2f1fd5c27e7dde6d4db089d902dbed7482d4f2263b1ac6bc8cd"
Jul 28 20:10:23.000044 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:22.999653    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f46b6dcfb-8p2mw"
Jul 28 20:10:23.039125 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:23.039081    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f46b6dcfb-8p2mw"]
Jul 28 20:10:23.048966 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:23.048933    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f46b6dcfb-8p2mw"]
Jul 28 20:10:23.339375 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:23.339328    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9be6a081-53cb-449d-a9d7-ecfa5a37a683" path="/var/lib/kubelet/pods/9be6a081-53cb-449d-a9d7-ecfa5a37a683/volumes"
Jul 28 20:10:25.807141 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:25.807109    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q9m9r"
Jul 28 20:10:27.868744 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:27.868679    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84685cbd4f-qwh9w" podUID="41f107b2-411f-4631-a6f0-59ef867edc19" containerName="console" containerID="cri-o://3f4718b3858062b9abd85b2fcefc22e521aabaadf65fecf3c6e46871b1997571" gracePeriod=15
Jul 28 20:10:28.025669 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.025548    2558 generic.go:356] "Generic (PLEG): container finished" podID="41f107b2-411f-4631-a6f0-59ef867edc19" containerID="3f4718b3858062b9abd85b2fcefc22e521aabaadf65fecf3c6e46871b1997571" exitCode=0
Jul 28 20:10:28.025669 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.025601    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84685cbd4f-qwh9w" event={"ID":"41f107b2-411f-4631-a6f0-59ef867edc19","Type":"ContainerDied","Data":"3f4718b3858062b9abd85b2fcefc22e521aabaadf65fecf3c6e46871b1997571"}
Jul 28 20:10:28.151545 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.151513    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:10:28.241598 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.241536    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-service-ca\") pod \"41f107b2-411f-4631-a6f0-59ef867edc19\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") "
Jul 28 20:10:28.241789 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.241621    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfg7r\" (UniqueName: \"kubernetes.io/projected/41f107b2-411f-4631-a6f0-59ef867edc19-kube-api-access-vfg7r\") pod \"41f107b2-411f-4631-a6f0-59ef867edc19\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") "
Jul 28 20:10:28.241789 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.241665    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-console-config\") pod \"41f107b2-411f-4631-a6f0-59ef867edc19\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") "
Jul 28 20:10:28.241789 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.241686    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-oauth-serving-cert\") pod \"41f107b2-411f-4631-a6f0-59ef867edc19\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") "
Jul 28 20:10:28.241789 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.241702    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41f107b2-411f-4631-a6f0-59ef867edc19-console-serving-cert\") pod \"41f107b2-411f-4631-a6f0-59ef867edc19\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") "
Jul 28 20:10:28.241789 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.241723    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41f107b2-411f-4631-a6f0-59ef867edc19-console-oauth-config\") pod \"41f107b2-411f-4631-a6f0-59ef867edc19\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") "
Jul 28 20:10:28.242093 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.241795    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-trusted-ca-bundle\") pod \"41f107b2-411f-4631-a6f0-59ef867edc19\" (UID: \"41f107b2-411f-4631-a6f0-59ef867edc19\") "
Jul 28 20:10:28.242093 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.242003    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-service-ca" (OuterVolumeSpecName: "service-ca") pod "41f107b2-411f-4631-a6f0-59ef867edc19" (UID: "41f107b2-411f-4631-a6f0-59ef867edc19"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:10:28.242197 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.242107    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "41f107b2-411f-4631-a6f0-59ef867edc19" (UID: "41f107b2-411f-4631-a6f0-59ef867edc19"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:10:28.242277 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.242221    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-oauth-serving-cert\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:28.242277 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.242239    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-service-ca\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:28.242363 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.242266    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-console-config" (OuterVolumeSpecName: "console-config") pod "41f107b2-411f-4631-a6f0-59ef867edc19" (UID: "41f107b2-411f-4631-a6f0-59ef867edc19"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:10:28.242363 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.242273    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "41f107b2-411f-4631-a6f0-59ef867edc19" (UID: "41f107b2-411f-4631-a6f0-59ef867edc19"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:10:28.243980 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.243951    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41f107b2-411f-4631-a6f0-59ef867edc19-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "41f107b2-411f-4631-a6f0-59ef867edc19" (UID: "41f107b2-411f-4631-a6f0-59ef867edc19"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:10:28.244136 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.244110    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41f107b2-411f-4631-a6f0-59ef867edc19-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "41f107b2-411f-4631-a6f0-59ef867edc19" (UID: "41f107b2-411f-4631-a6f0-59ef867edc19"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:10:28.244194 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.244130    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41f107b2-411f-4631-a6f0-59ef867edc19-kube-api-access-vfg7r" (OuterVolumeSpecName: "kube-api-access-vfg7r") pod "41f107b2-411f-4631-a6f0-59ef867edc19" (UID: "41f107b2-411f-4631-a6f0-59ef867edc19"). InnerVolumeSpecName "kube-api-access-vfg7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:10:28.343685 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.343645    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-trusted-ca-bundle\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:28.343685 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.343677    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfg7r\" (UniqueName: \"kubernetes.io/projected/41f107b2-411f-4631-a6f0-59ef867edc19-kube-api-access-vfg7r\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:28.343685 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.343688    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41f107b2-411f-4631-a6f0-59ef867edc19-console-config\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:28.343918 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.343697    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41f107b2-411f-4631-a6f0-59ef867edc19-console-serving-cert\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:28.343918 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:28.343709    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41f107b2-411f-4631-a6f0-59ef867edc19-console-oauth-config\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:10:29.031698 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:29.031675    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84685cbd4f-qwh9w"
Jul 28 20:10:29.032088 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:29.031671    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84685cbd4f-qwh9w" event={"ID":"41f107b2-411f-4631-a6f0-59ef867edc19","Type":"ContainerDied","Data":"e52d48c952e9a78f1bd72fed75e0d0a7c2ff8dcb0eb991945715bdfb0c88836b"}
Jul 28 20:10:29.032088 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:29.031799    2558 scope.go:117] "RemoveContainer" containerID="3f4718b3858062b9abd85b2fcefc22e521aabaadf65fecf3c6e46871b1997571"
Jul 28 20:10:29.053612 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:29.053552    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84685cbd4f-qwh9w"]
Jul 28 20:10:29.058635 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:29.058602    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84685cbd4f-qwh9w"]
Jul 28 20:10:29.342391 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:29.342306    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41f107b2-411f-4631-a6f0-59ef867edc19" path="/var/lib/kubelet/pods/41f107b2-411f-4631-a6f0-59ef867edc19/volumes"
Jul 28 20:10:36.688657 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:36.688626    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:10:36.692631 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:36.692607    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-67bf6bb647-f58q9"
Jul 28 20:10:58.429336 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:58.429297    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:10:58.449336 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:58.449303    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:10:59.176077 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:10:59.176042    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:12:56.168737 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.168702    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-slrm7"]
Jul 28 20:12:56.169208 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.169047    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9be6a081-53cb-449d-a9d7-ecfa5a37a683" containerName="console"
Jul 28 20:12:56.169208 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.169058    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9be6a081-53cb-449d-a9d7-ecfa5a37a683" containerName="console"
Jul 28 20:12:56.169208 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.169068    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41f107b2-411f-4631-a6f0-59ef867edc19" containerName="console"
Jul 28 20:12:56.169208 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.169074    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="41f107b2-411f-4631-a6f0-59ef867edc19" containerName="console"
Jul 28 20:12:56.169208 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.169123    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9be6a081-53cb-449d-a9d7-ecfa5a37a683" containerName="console"
Jul 28 20:12:56.169208 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.169132    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="41f107b2-411f-4631-a6f0-59ef867edc19" containerName="console"
Jul 28 20:12:56.171148 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.171128    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-slrm7"
Jul 28 20:12:56.176398 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.176373    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 20:12:56.177233 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.177201    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 20:12:56.177233 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.177201    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-54pvw\""
Jul 28 20:12:56.184124 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.184096    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-slrm7"]
Jul 28 20:12:56.270670 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.270633    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36f5ec0-e244-453d-97e8-3cc7d58463a6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-slrm7\" (UID: \"f36f5ec0-e244-453d-97e8-3cc7d58463a6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-slrm7"
Jul 28 20:12:56.270857 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.270733    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcfjk\" (UniqueName: \"kubernetes.io/projected/f36f5ec0-e244-453d-97e8-3cc7d58463a6-kube-api-access-dcfjk\") pod \"cert-manager-webhook-6486ffd997-slrm7\" (UID: \"f36f5ec0-e244-453d-97e8-3cc7d58463a6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-slrm7"
Jul 28 20:12:56.372095 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.372059    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcfjk\" (UniqueName: \"kubernetes.io/projected/f36f5ec0-e244-453d-97e8-3cc7d58463a6-kube-api-access-dcfjk\") pod \"cert-manager-webhook-6486ffd997-slrm7\" (UID: \"f36f5ec0-e244-453d-97e8-3cc7d58463a6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-slrm7"
Jul 28 20:12:56.372279 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.372112    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36f5ec0-e244-453d-97e8-3cc7d58463a6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-slrm7\" (UID: \"f36f5ec0-e244-453d-97e8-3cc7d58463a6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-slrm7"
Jul 28 20:12:56.384541 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.384507    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcfjk\" (UniqueName: \"kubernetes.io/projected/f36f5ec0-e244-453d-97e8-3cc7d58463a6-kube-api-access-dcfjk\") pod \"cert-manager-webhook-6486ffd997-slrm7\" (UID: \"f36f5ec0-e244-453d-97e8-3cc7d58463a6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-slrm7"
Jul 28 20:12:56.384843 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.384823    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36f5ec0-e244-453d-97e8-3cc7d58463a6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-slrm7\" (UID: \"f36f5ec0-e244-453d-97e8-3cc7d58463a6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-slrm7"
Jul 28 20:12:56.488127 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.488034    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-slrm7"
Jul 28 20:12:56.620076 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.620048    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-slrm7"]
Jul 28 20:12:56.623117 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:12:56.623072    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf36f5ec0_e244_453d_97e8_3cc7d58463a6.slice/crio-bf2426118019f6793450e9864981c9735afbb8e43c7ecdc70a129efd68cb98e1 WatchSource:0}: Error finding container bf2426118019f6793450e9864981c9735afbb8e43c7ecdc70a129efd68cb98e1: Status 404 returned error can't find the container with id bf2426118019f6793450e9864981c9735afbb8e43c7ecdc70a129efd68cb98e1
Jul 28 20:12:56.644990 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:12:56.644956    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-slrm7" event={"ID":"f36f5ec0-e244-453d-97e8-3cc7d58463a6","Type":"ContainerStarted","Data":"bf2426118019f6793450e9864981c9735afbb8e43c7ecdc70a129efd68cb98e1"}
Jul 28 20:13:00.667506 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:00.667464    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-slrm7" event={"ID":"f36f5ec0-e244-453d-97e8-3cc7d58463a6","Type":"ContainerStarted","Data":"1a0649c511ee042fa586eb5d09e040a90a0ce50b914e83844f6d45bfacfac75a"}
Jul 28 20:13:00.667907 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:00.667592    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-slrm7"
Jul 28 20:13:00.690461 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:00.690408    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-slrm7" podStartSLOduration=1.33042739 podStartE2EDuration="4.69038958s" totalImagesPullingTime="3.35996219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:13:00.689783501 +0000 UTC m=+257.931876723" watchObservedRunningTime="2026-07-28 20:13:00.69038958 +0000 UTC m=+257.932482803"
Jul 28 20:13:06.673623 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:06.673530    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-slrm7"
Jul 28 20:13:09.287228 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.287177    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q"]
Jul 28 20:13:09.292509 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.292489    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q"
Jul 28 20:13:09.295044 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.295019    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 20:13:09.295185 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.295133    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 20:13:09.296013 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.295973    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-ph9c6\""
Jul 28 20:13:09.298482 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.298455    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q"]
Jul 28 20:13:09.390071 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.390025    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c45rr\" (UniqueName: \"kubernetes.io/projected/74f3fecf-92b1-4a04-ae8d-c71ccdd8f973-kube-api-access-c45rr\") pod \"openshift-lws-operator-fd8ccff4c-9r25q\" (UID: \"74f3fecf-92b1-4a04-ae8d-c71ccdd8f973\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q"
Jul 28 20:13:09.390256 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.390130    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74f3fecf-92b1-4a04-ae8d-c71ccdd8f973-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9r25q\" (UID: \"74f3fecf-92b1-4a04-ae8d-c71ccdd8f973\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q"
Jul 28 20:13:09.491469 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.491426    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74f3fecf-92b1-4a04-ae8d-c71ccdd8f973-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9r25q\" (UID: \"74f3fecf-92b1-4a04-ae8d-c71ccdd8f973\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q"
Jul 28 20:13:09.491702 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.491495    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45rr\" (UniqueName: \"kubernetes.io/projected/74f3fecf-92b1-4a04-ae8d-c71ccdd8f973-kube-api-access-c45rr\") pod \"openshift-lws-operator-fd8ccff4c-9r25q\" (UID: \"74f3fecf-92b1-4a04-ae8d-c71ccdd8f973\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q"
Jul 28 20:13:09.491864 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.491842    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74f3fecf-92b1-4a04-ae8d-c71ccdd8f973-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9r25q\" (UID: \"74f3fecf-92b1-4a04-ae8d-c71ccdd8f973\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q"
Jul 28 20:13:09.500745 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.500716    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c45rr\" (UniqueName: \"kubernetes.io/projected/74f3fecf-92b1-4a04-ae8d-c71ccdd8f973-kube-api-access-c45rr\") pod \"openshift-lws-operator-fd8ccff4c-9r25q\" (UID: \"74f3fecf-92b1-4a04-ae8d-c71ccdd8f973\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q"
Jul 28 20:13:09.603270 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.603174    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q"
Jul 28 20:13:09.737683 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:09.737657    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q"]
Jul 28 20:13:09.740045 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:13:09.740019    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74f3fecf_92b1_4a04_ae8d_c71ccdd8f973.slice/crio-23953acd7f05079e02a7d70e81ab77c625ff62348a58a609eca992159732d7c1 WatchSource:0}: Error finding container 23953acd7f05079e02a7d70e81ab77c625ff62348a58a609eca992159732d7c1: Status 404 returned error can't find the container with id 23953acd7f05079e02a7d70e81ab77c625ff62348a58a609eca992159732d7c1
Jul 28 20:13:10.709813 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:10.709782    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q" event={"ID":"74f3fecf-92b1-4a04-ae8d-c71ccdd8f973","Type":"ContainerStarted","Data":"23953acd7f05079e02a7d70e81ab77c625ff62348a58a609eca992159732d7c1"}
Jul 28 20:13:13.726253 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:13.726146    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q" event={"ID":"74f3fecf-92b1-4a04-ae8d-c71ccdd8f973","Type":"ContainerStarted","Data":"c6a7491ea11bfafacced56dc96d41d47e3cfd72ae0c1a1a78aa09cd5c75593a0"}
Jul 28 20:13:13.746327 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:13.746268    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9r25q" podStartSLOduration=1.020372781 podStartE2EDuration="4.746250962s" totalImagesPullingTime="3.725878181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:13:13.744988495 +0000 UTC m=+270.987081719" watchObservedRunningTime="2026-07-28 20:13:13.746250962 +0000 UTC m=+270.988344186"
Jul 28 20:13:22.565007 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.564975    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"]
Jul 28 20:13:22.569161 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.569140    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:22.575102 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.575079    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 20:13:22.576364 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.576334    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 20:13:22.576497 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.576479    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5thc9\""
Jul 28 20:13:22.576605 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.576590    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 20:13:22.588486 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.588455    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"]
Jul 28 20:13:22.605717 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.605679    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/00d9d98b-0bac-4480-ba10-6361452a37a8-manager-config\") pod \"lws-controller-manager-6f8978bd54-bnj44\" (UID: \"00d9d98b-0bac-4480-ba10-6361452a37a8\") " pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:22.605895 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.605733    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47vvr\" (UniqueName: \"kubernetes.io/projected/00d9d98b-0bac-4480-ba10-6361452a37a8-kube-api-access-47vvr\") pod \"lws-controller-manager-6f8978bd54-bnj44\" (UID: \"00d9d98b-0bac-4480-ba10-6361452a37a8\") " pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:22.605895 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.605762    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/00d9d98b-0bac-4480-ba10-6361452a37a8-metrics-cert\") pod \"lws-controller-manager-6f8978bd54-bnj44\" (UID: \"00d9d98b-0bac-4480-ba10-6361452a37a8\") " pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:22.605895 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.605807    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00d9d98b-0bac-4480-ba10-6361452a37a8-cert\") pod \"lws-controller-manager-6f8978bd54-bnj44\" (UID: \"00d9d98b-0bac-4480-ba10-6361452a37a8\") " pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:22.706231 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.706192    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/00d9d98b-0bac-4480-ba10-6361452a37a8-metrics-cert\") pod \"lws-controller-manager-6f8978bd54-bnj44\" (UID: \"00d9d98b-0bac-4480-ba10-6361452a37a8\") " pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:22.706231 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.706239    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00d9d98b-0bac-4480-ba10-6361452a37a8-cert\") pod \"lws-controller-manager-6f8978bd54-bnj44\" (UID: \"00d9d98b-0bac-4480-ba10-6361452a37a8\") " pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:22.706473 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.706284    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/00d9d98b-0bac-4480-ba10-6361452a37a8-manager-config\") pod \"lws-controller-manager-6f8978bd54-bnj44\" (UID: \"00d9d98b-0bac-4480-ba10-6361452a37a8\") " pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:22.706473 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.706316    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47vvr\" (UniqueName: \"kubernetes.io/projected/00d9d98b-0bac-4480-ba10-6361452a37a8-kube-api-access-47vvr\") pod \"lws-controller-manager-6f8978bd54-bnj44\" (UID: \"00d9d98b-0bac-4480-ba10-6361452a37a8\") " pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:22.707017 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.706995    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/00d9d98b-0bac-4480-ba10-6361452a37a8-manager-config\") pod \"lws-controller-manager-6f8978bd54-bnj44\" (UID: \"00d9d98b-0bac-4480-ba10-6361452a37a8\") " pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:22.708835 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.708815    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00d9d98b-0bac-4480-ba10-6361452a37a8-cert\") pod \"lws-controller-manager-6f8978bd54-bnj44\" (UID: \"00d9d98b-0bac-4480-ba10-6361452a37a8\") " pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:22.708925 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.708853    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/00d9d98b-0bac-4480-ba10-6361452a37a8-metrics-cert\") pod \"lws-controller-manager-6f8978bd54-bnj44\" (UID: \"00d9d98b-0bac-4480-ba10-6361452a37a8\") " pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:22.721975 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.721947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47vvr\" (UniqueName: \"kubernetes.io/projected/00d9d98b-0bac-4480-ba10-6361452a37a8-kube-api-access-47vvr\") pod \"lws-controller-manager-6f8978bd54-bnj44\" (UID: \"00d9d98b-0bac-4480-ba10-6361452a37a8\") " pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:22.879026 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:22.878938    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:23.026886 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:23.026849    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"]
Jul 28 20:13:23.031019 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:13:23.030991    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00d9d98b_0bac_4480_ba10_6361452a37a8.slice/crio-99c2c4c91436d574db6e115e887b729f149850e302235b68cab95f0ea85b2bdc WatchSource:0}: Error finding container 99c2c4c91436d574db6e115e887b729f149850e302235b68cab95f0ea85b2bdc: Status 404 returned error can't find the container with id 99c2c4c91436d574db6e115e887b729f149850e302235b68cab95f0ea85b2bdc
Jul 28 20:13:23.770741 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:23.770704    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44" event={"ID":"00d9d98b-0bac-4480-ba10-6361452a37a8","Type":"ContainerStarted","Data":"99c2c4c91436d574db6e115e887b729f149850e302235b68cab95f0ea85b2bdc"}
Jul 28 20:13:24.776770 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:24.776729    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44" event={"ID":"00d9d98b-0bac-4480-ba10-6361452a37a8","Type":"ContainerStarted","Data":"23fb2d3a75932798c7da74f68f1d8313c8eb62f8f3d46d2148925bd98fdb0048"}
Jul 28 20:13:24.777177 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:24.776844    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:24.805993 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:24.805881    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44" podStartSLOduration=1.292126949 podStartE2EDuration="2.805864939s" totalImagesPullingTime="1.51373799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:13:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:13:24.805685893 +0000 UTC m=+282.047779116" watchObservedRunningTime="2026-07-28 20:13:24.805864939 +0000 UTC m=+282.047958162"
Jul 28 20:13:29.314473 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.314437    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"]
Jul 28 20:13:29.317074 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.317052    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.323826 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.323794    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 20:13:29.323961 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.323880    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-h57cb\""
Jul 28 20:13:29.325845 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.325816    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 20:13:29.325994 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.325969    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 20:13:29.326218 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.326194    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 20:13:29.340604 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.340548    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"]
Jul 28 20:13:29.366542 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.366506    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/18c742c1-226f-4208-a374-e7cbec444d18-manifests\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.366542 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.366546    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18c742c1-226f-4208-a374-e7cbec444d18-tmp\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.366786 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.366597    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ljfp\" (UniqueName: \"kubernetes.io/projected/18c742c1-226f-4208-a374-e7cbec444d18-kube-api-access-7ljfp\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.366786 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.366645    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18c742c1-226f-4208-a374-e7cbec444d18-webhook-cert\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.366786 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.366708    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/18c742c1-226f-4208-a374-e7cbec444d18-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.467324 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.467294    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18c742c1-226f-4208-a374-e7cbec444d18-webhook-cert\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.467324 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.467336    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/18c742c1-226f-4208-a374-e7cbec444d18-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.467640 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.467398    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/18c742c1-226f-4208-a374-e7cbec444d18-manifests\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.467640 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.467415    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18c742c1-226f-4208-a374-e7cbec444d18-tmp\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.467640 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.467433    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ljfp\" (UniqueName: \"kubernetes.io/projected/18c742c1-226f-4208-a374-e7cbec444d18-kube-api-access-7ljfp\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.467879 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.467853    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/18c742c1-226f-4208-a374-e7cbec444d18-manifests\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.469785 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.469756    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18c742c1-226f-4208-a374-e7cbec444d18-tmp\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.469926 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.469839    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18c742c1-226f-4208-a374-e7cbec444d18-webhook-cert\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.469926 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.469885    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/18c742c1-226f-4208-a374-e7cbec444d18-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.489692 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.489664    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ljfp\" (UniqueName: \"kubernetes.io/projected/18c742c1-226f-4208-a374-e7cbec444d18-kube-api-access-7ljfp\") pod \"opendatahub-operator-controller-manager-74ff7fbd5d-kbj97\" (UID: \"18c742c1-226f-4208-a374-e7cbec444d18\") " pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.628819 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.628726    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:29.772329 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.772223    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"]
Jul 28 20:13:29.775323 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:13:29.775296    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c742c1_226f_4208_a374_e7cbec444d18.slice/crio-d6ed7d2b4c16e930ad1db416d6befdfc60eaac6594e73aaff2b28bd42ce50ecf WatchSource:0}: Error finding container d6ed7d2b4c16e930ad1db416d6befdfc60eaac6594e73aaff2b28bd42ce50ecf: Status 404 returned error can't find the container with id d6ed7d2b4c16e930ad1db416d6befdfc60eaac6594e73aaff2b28bd42ce50ecf
Jul 28 20:13:29.798750 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:29.798714    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97" event={"ID":"18c742c1-226f-4208-a374-e7cbec444d18","Type":"ContainerStarted","Data":"d6ed7d2b4c16e930ad1db416d6befdfc60eaac6594e73aaff2b28bd42ce50ecf"}
Jul 28 20:13:33.823109 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:33.823067    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97" event={"ID":"18c742c1-226f-4208-a374-e7cbec444d18","Type":"ContainerStarted","Data":"77dda03c532439276f5c9e6a1144a437c348e17a090075429d3ca7b3235435b1"}
Jul 28 20:13:34.830146 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:34.830108    2558 generic.go:356] "Generic (PLEG): container finished" podID="18c742c1-226f-4208-a374-e7cbec444d18" containerID="77dda03c532439276f5c9e6a1144a437c348e17a090075429d3ca7b3235435b1" exitCode=0
Jul 28 20:13:34.830644 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:34.830205    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97" event={"ID":"18c742c1-226f-4208-a374-e7cbec444d18","Type":"ContainerDied","Data":"77dda03c532439276f5c9e6a1144a437c348e17a090075429d3ca7b3235435b1"}
Jul 28 20:13:35.783071 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:35.783039    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6f8978bd54-bnj44"
Jul 28 20:13:35.838429 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:35.838395    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97" event={"ID":"18c742c1-226f-4208-a374-e7cbec444d18","Type":"ContainerStarted","Data":"25a77ce723abac760ef9239d046bbd6d1ddb3b407efc84b75c390837ddf18c75"}
Jul 28 20:13:35.838843 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:35.838523    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:35.860654 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:35.860603    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97" podStartSLOduration=2.234718646 podStartE2EDuration="6.860585195s" totalImagesPullingTime="3.562888043s" totalInitContainerRuntime="1.062978506s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:13:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:13:35.858901898 +0000 UTC m=+293.100995124" watchObservedRunningTime="2026-07-28 20:13:35.860585195 +0000 UTC m=+293.102678418"
Jul 28 20:13:43.139984 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:43.139949    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:13:43.145929 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:43.145904    2558 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 20:13:43.146608 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:43.146542    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:13:46.845485 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:46.845452    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74ff7fbd5d-kbj97"
Jul 28 20:13:49.094116 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.094080    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"]
Jul 28 20:13:49.096479 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.096462    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"
Jul 28 20:13:49.099519 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.099494    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 20:13:49.099519 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.099505    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 20:13:49.099736 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.099580    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-v72ch\""
Jul 28 20:13:49.099736 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.099642    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 20:13:49.099830 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.099762    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 20:13:49.111379 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.111353    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"]
Jul 28 20:13:49.248953 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.248914    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/115f81c7-6e64-414a-aa77-fac6a2dc2224-tmp\") pod \"kube-auth-proxy-5bb564c89-wtzwv\" (UID: \"115f81c7-6e64-414a-aa77-fac6a2dc2224\") " pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"
Jul 28 20:13:49.248953 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.248965    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9wzv\" (UniqueName: \"kubernetes.io/projected/115f81c7-6e64-414a-aa77-fac6a2dc2224-kube-api-access-m9wzv\") pod \"kube-auth-proxy-5bb564c89-wtzwv\" (UID: \"115f81c7-6e64-414a-aa77-fac6a2dc2224\") " pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"
Jul 28 20:13:49.249203 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.249036    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/115f81c7-6e64-414a-aa77-fac6a2dc2224-tls-certs\") pod \"kube-auth-proxy-5bb564c89-wtzwv\" (UID: \"115f81c7-6e64-414a-aa77-fac6a2dc2224\") " pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"
Jul 28 20:13:49.349935 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.349842    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/115f81c7-6e64-414a-aa77-fac6a2dc2224-tmp\") pod \"kube-auth-proxy-5bb564c89-wtzwv\" (UID: \"115f81c7-6e64-414a-aa77-fac6a2dc2224\") " pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"
Jul 28 20:13:49.349935 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.349892    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9wzv\" (UniqueName: \"kubernetes.io/projected/115f81c7-6e64-414a-aa77-fac6a2dc2224-kube-api-access-m9wzv\") pod \"kube-auth-proxy-5bb564c89-wtzwv\" (UID: \"115f81c7-6e64-414a-aa77-fac6a2dc2224\") " pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"
Jul 28 20:13:49.349935 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.349926    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/115f81c7-6e64-414a-aa77-fac6a2dc2224-tls-certs\") pod \"kube-auth-proxy-5bb564c89-wtzwv\" (UID: \"115f81c7-6e64-414a-aa77-fac6a2dc2224\") " pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"
Jul 28 20:13:49.350177 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:13:49.350049    2558 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 28 20:13:49.350177 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:13:49.350110    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/115f81c7-6e64-414a-aa77-fac6a2dc2224-tls-certs podName:115f81c7-6e64-414a-aa77-fac6a2dc2224 nodeName:}" failed. No retries permitted until 2026-07-28 20:13:49.850089164 +0000 UTC m=+307.092182365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/115f81c7-6e64-414a-aa77-fac6a2dc2224-tls-certs") pod "kube-auth-proxy-5bb564c89-wtzwv" (UID: "115f81c7-6e64-414a-aa77-fac6a2dc2224") : secret "kube-auth-proxy-tls" not found
Jul 28 20:13:49.352134 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.352107    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/115f81c7-6e64-414a-aa77-fac6a2dc2224-tmp\") pod \"kube-auth-proxy-5bb564c89-wtzwv\" (UID: \"115f81c7-6e64-414a-aa77-fac6a2dc2224\") " pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"
Jul 28 20:13:49.362251 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.362224    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9wzv\" (UniqueName: \"kubernetes.io/projected/115f81c7-6e64-414a-aa77-fac6a2dc2224-kube-api-access-m9wzv\") pod \"kube-auth-proxy-5bb564c89-wtzwv\" (UID: \"115f81c7-6e64-414a-aa77-fac6a2dc2224\") " pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"
Jul 28 20:13:49.854780 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.854733    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/115f81c7-6e64-414a-aa77-fac6a2dc2224-tls-certs\") pod \"kube-auth-proxy-5bb564c89-wtzwv\" (UID: \"115f81c7-6e64-414a-aa77-fac6a2dc2224\") " pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"
Jul 28 20:13:49.857197 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:49.857163    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/115f81c7-6e64-414a-aa77-fac6a2dc2224-tls-certs\") pod \"kube-auth-proxy-5bb564c89-wtzwv\" (UID: \"115f81c7-6e64-414a-aa77-fac6a2dc2224\") " pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"
Jul 28 20:13:50.007276 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:50.007238    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"
Jul 28 20:13:50.181632 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:50.181600    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv"]
Jul 28 20:13:50.183866 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:13:50.183824    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod115f81c7_6e64_414a_aa77_fac6a2dc2224.slice/crio-e0d768cdbe36db84b6cfcb5018f7e09843a1ca0500f25f5f2165caff55460ccf WatchSource:0}: Error finding container e0d768cdbe36db84b6cfcb5018f7e09843a1ca0500f25f5f2165caff55460ccf: Status 404 returned error can't find the container with id e0d768cdbe36db84b6cfcb5018f7e09843a1ca0500f25f5f2165caff55460ccf
Jul 28 20:13:50.186023 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:50.185997    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:13:50.909393 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:50.909355    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv" event={"ID":"115f81c7-6e64-414a-aa77-fac6a2dc2224","Type":"ContainerStarted","Data":"e0d768cdbe36db84b6cfcb5018f7e09843a1ca0500f25f5f2165caff55460ccf"}
Jul 28 20:13:52.921675 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:52.921601    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv" event={"ID":"115f81c7-6e64-414a-aa77-fac6a2dc2224","Type":"ContainerStarted","Data":"f27b84ad8fd7c32c06bf39aaa59a8d19f2e854e15499118bdfe55ea0d58530b1"}
Jul 28 20:13:52.939401 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:13:52.939340    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5bb564c89-wtzwv" podStartSLOduration=1.284861088 podStartE2EDuration="3.939322644s" totalImagesPullingTime="2.654461556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:13:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:13:52.938043805 +0000 UTC m=+310.180137029" watchObservedRunningTime="2026-07-28 20:13:52.939322644 +0000 UTC m=+310.181415866"
Jul 28 20:15:16.994552 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:16.994512    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74dbfd7fbd-mg6k6"]
Jul 28 20:15:16.997179 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:16.997154    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:16.999610 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:16.999582    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 20:15:17.000770 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.000748    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9zrj5\""
Jul 28 20:15:17.000770 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.000763    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 20:15:17.000957 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.000793    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 20:15:17.000957 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.000804    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 20:15:17.000957 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.000862    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 20:15:17.000957 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.000954    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 20:15:17.001248 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.001233    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 20:15:17.005525 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.005506    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 20:15:17.007359 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.007334    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74dbfd7fbd-mg6k6"]
Jul 28 20:15:17.145609 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.145541    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80390a5f-7218-44c8-b025-6d2c0c6d542a-service-ca\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.145609 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.145603    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80390a5f-7218-44c8-b025-6d2c0c6d542a-console-oauth-config\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.145889 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.145623    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80390a5f-7218-44c8-b025-6d2c0c6d542a-oauth-serving-cert\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.145889 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.145745    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80390a5f-7218-44c8-b025-6d2c0c6d542a-console-config\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.145889 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.145826    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbqdx\" (UniqueName: \"kubernetes.io/projected/80390a5f-7218-44c8-b025-6d2c0c6d542a-kube-api-access-vbqdx\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.145889 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.145869    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80390a5f-7218-44c8-b025-6d2c0c6d542a-console-serving-cert\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.146048 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.145907    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80390a5f-7218-44c8-b025-6d2c0c6d542a-trusted-ca-bundle\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.246820 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.246720    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80390a5f-7218-44c8-b025-6d2c0c6d542a-service-ca\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.246820 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.246765    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80390a5f-7218-44c8-b025-6d2c0c6d542a-console-oauth-config\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.246820 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.246786    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80390a5f-7218-44c8-b025-6d2c0c6d542a-oauth-serving-cert\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.246820 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.246825    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80390a5f-7218-44c8-b025-6d2c0c6d542a-console-config\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.247170 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.246865    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbqdx\" (UniqueName: \"kubernetes.io/projected/80390a5f-7218-44c8-b025-6d2c0c6d542a-kube-api-access-vbqdx\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.247170 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.246889    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80390a5f-7218-44c8-b025-6d2c0c6d542a-console-serving-cert\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.247170 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.246918    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80390a5f-7218-44c8-b025-6d2c0c6d542a-trusted-ca-bundle\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.247682 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.247656    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80390a5f-7218-44c8-b025-6d2c0c6d542a-console-config\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.247799 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.247677    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80390a5f-7218-44c8-b025-6d2c0c6d542a-oauth-serving-cert\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.247799 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.247772    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80390a5f-7218-44c8-b025-6d2c0c6d542a-service-ca\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.247913 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.247904    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80390a5f-7218-44c8-b025-6d2c0c6d542a-trusted-ca-bundle\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.249342 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.249323    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80390a5f-7218-44c8-b025-6d2c0c6d542a-console-serving-cert\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.249427 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.249374    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80390a5f-7218-44c8-b025-6d2c0c6d542a-console-oauth-config\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.256145 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.256122    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbqdx\" (UniqueName: \"kubernetes.io/projected/80390a5f-7218-44c8-b025-6d2c0c6d542a-kube-api-access-vbqdx\") pod \"console-74dbfd7fbd-mg6k6\" (UID: \"80390a5f-7218-44c8-b025-6d2c0c6d542a\") " pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.307775 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.307742    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:17.447044 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:17.447012    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74dbfd7fbd-mg6k6"]
Jul 28 20:15:17.449589 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:15:17.449532    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80390a5f_7218_44c8_b025_6d2c0c6d542a.slice/crio-c8b3e4e66d9bcf05a21f5b45c5ac2bd897af57d968a71e4f62a8729b2e2fd2d0 WatchSource:0}: Error finding container c8b3e4e66d9bcf05a21f5b45c5ac2bd897af57d968a71e4f62a8729b2e2fd2d0: Status 404 returned error can't find the container with id c8b3e4e66d9bcf05a21f5b45c5ac2bd897af57d968a71e4f62a8729b2e2fd2d0
Jul 28 20:15:18.316956 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:18.316918    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74dbfd7fbd-mg6k6" event={"ID":"80390a5f-7218-44c8-b025-6d2c0c6d542a","Type":"ContainerStarted","Data":"4e3c61384e29f9fe53d65dc7a8b899c896ede6ca36455ef1a1409c26615e06ca"}
Jul 28 20:15:18.317382 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:18.316974    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74dbfd7fbd-mg6k6" event={"ID":"80390a5f-7218-44c8-b025-6d2c0c6d542a","Type":"ContainerStarted","Data":"c8b3e4e66d9bcf05a21f5b45c5ac2bd897af57d968a71e4f62a8729b2e2fd2d0"}
Jul 28 20:15:18.335269 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:18.335209    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74dbfd7fbd-mg6k6" podStartSLOduration=2.335187897 podStartE2EDuration="2.335187897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:15:18.334808761 +0000 UTC m=+395.576901986" watchObservedRunningTime="2026-07-28 20:15:18.335187897 +0000 UTC m=+395.577281121"
Jul 28 20:15:27.308402 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:27.308360    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:27.308840 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:27.308427    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:27.313205 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:27.313179    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:27.361605 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:27.361575    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74dbfd7fbd-mg6k6"
Jul 28 20:15:46.262285 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.262247    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"]
Jul 28 20:15:46.264643 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.264623    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"
Jul 28 20:15:46.267174 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.267143    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t9j99\""
Jul 28 20:15:46.267319 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.267172    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 28 20:15:46.267319 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.267221    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 20:15:46.268262 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.268241    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 20:15:46.268262 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.268259    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 28 20:15:46.277471 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.277442    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"]
Jul 28 20:15:46.313104 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.313063    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f9febd1-7e8a-40a9-829f-4151279e95dc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kzlwd\" (UID: \"7f9febd1-7e8a-40a9-829f-4151279e95dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"
Jul 28 20:15:46.313296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.313161    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7v6p\" (UniqueName: \"kubernetes.io/projected/7f9febd1-7e8a-40a9-829f-4151279e95dc-kube-api-access-t7v6p\") pod \"kuadrant-console-plugin-66b5774985-kzlwd\" (UID: \"7f9febd1-7e8a-40a9-829f-4151279e95dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"
Jul 28 20:15:46.313296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.313192    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7f9febd1-7e8a-40a9-829f-4151279e95dc-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kzlwd\" (UID: \"7f9febd1-7e8a-40a9-829f-4151279e95dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"
Jul 28 20:15:46.414021 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.413984    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f9febd1-7e8a-40a9-829f-4151279e95dc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kzlwd\" (UID: \"7f9febd1-7e8a-40a9-829f-4151279e95dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"
Jul 28 20:15:46.414218 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.414077    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7v6p\" (UniqueName: \"kubernetes.io/projected/7f9febd1-7e8a-40a9-829f-4151279e95dc-kube-api-access-t7v6p\") pod \"kuadrant-console-plugin-66b5774985-kzlwd\" (UID: \"7f9febd1-7e8a-40a9-829f-4151279e95dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"
Jul 28 20:15:46.414218 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.414106    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7f9febd1-7e8a-40a9-829f-4151279e95dc-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kzlwd\" (UID: \"7f9febd1-7e8a-40a9-829f-4151279e95dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"
Jul 28 20:15:46.414218 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:15:46.414134    2558 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 28 20:15:46.414218 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:15:46.414207    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f9febd1-7e8a-40a9-829f-4151279e95dc-plugin-serving-cert podName:7f9febd1-7e8a-40a9-829f-4151279e95dc nodeName:}" failed. No retries permitted until 2026-07-28 20:15:46.914188954 +0000 UTC m=+424.156282158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/7f9febd1-7e8a-40a9-829f-4151279e95dc-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-kzlwd" (UID: "7f9febd1-7e8a-40a9-829f-4151279e95dc") : secret "plugin-serving-cert" not found
Jul 28 20:15:46.414741 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.414724    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7f9febd1-7e8a-40a9-829f-4151279e95dc-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kzlwd\" (UID: \"7f9febd1-7e8a-40a9-829f-4151279e95dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"
Jul 28 20:15:46.422330 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.422304    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7v6p\" (UniqueName: \"kubernetes.io/projected/7f9febd1-7e8a-40a9-829f-4151279e95dc-kube-api-access-t7v6p\") pod \"kuadrant-console-plugin-66b5774985-kzlwd\" (UID: \"7f9febd1-7e8a-40a9-829f-4151279e95dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"
Jul 28 20:15:46.919300 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.919254    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f9febd1-7e8a-40a9-829f-4151279e95dc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kzlwd\" (UID: \"7f9febd1-7e8a-40a9-829f-4151279e95dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"
Jul 28 20:15:46.921796 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:46.921769    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f9febd1-7e8a-40a9-829f-4151279e95dc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kzlwd\" (UID: \"7f9febd1-7e8a-40a9-829f-4151279e95dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"
Jul 28 20:15:47.174235 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:47.174141    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"
Jul 28 20:15:47.312885 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:47.312839    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd"]
Jul 28 20:15:47.316759 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:15:47.316722    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f9febd1_7e8a_40a9_829f_4151279e95dc.slice/crio-7c994fd26219e1ad34cb1b54e2a63228a9292961efc5ac812646180b26cff025 WatchSource:0}: Error finding container 7c994fd26219e1ad34cb1b54e2a63228a9292961efc5ac812646180b26cff025: Status 404 returned error can't find the container with id 7c994fd26219e1ad34cb1b54e2a63228a9292961efc5ac812646180b26cff025
Jul 28 20:15:47.452669 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:47.452547    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd" event={"ID":"7f9febd1-7e8a-40a9-829f-4151279e95dc","Type":"ContainerStarted","Data":"7c994fd26219e1ad34cb1b54e2a63228a9292961efc5ac812646180b26cff025"}
Jul 28 20:15:57.415332 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:57.415282    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"]
Jul 28 20:15:57.430846 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:57.430807    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"
Jul 28 20:15:57.433847 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:57.433788    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4qq2g\""
Jul 28 20:15:57.439159 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:57.439129    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"]
Jul 28 20:15:57.528388 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:57.528345    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f9b62c3-bb42-4ed5-b734-48590bf42ed5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gl6jb\" (UID: \"2f9b62c3-bb42-4ed5-b734-48590bf42ed5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"
Jul 28 20:15:57.528591 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:57.528405    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrcnk\" (UniqueName: \"kubernetes.io/projected/2f9b62c3-bb42-4ed5-b734-48590bf42ed5-kube-api-access-vrcnk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gl6jb\" (UID: \"2f9b62c3-bb42-4ed5-b734-48590bf42ed5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"
Jul 28 20:15:57.630010 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:57.629972    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f9b62c3-bb42-4ed5-b734-48590bf42ed5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gl6jb\" (UID: \"2f9b62c3-bb42-4ed5-b734-48590bf42ed5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"
Jul 28 20:15:57.630010 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:57.630016    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrcnk\" (UniqueName: \"kubernetes.io/projected/2f9b62c3-bb42-4ed5-b734-48590bf42ed5-kube-api-access-vrcnk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gl6jb\" (UID: \"2f9b62c3-bb42-4ed5-b734-48590bf42ed5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"
Jul 28 20:15:57.630435 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:57.630411    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f9b62c3-bb42-4ed5-b734-48590bf42ed5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gl6jb\" (UID: \"2f9b62c3-bb42-4ed5-b734-48590bf42ed5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"
Jul 28 20:15:57.638093 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:57.638063    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrcnk\" (UniqueName: \"kubernetes.io/projected/2f9b62c3-bb42-4ed5-b734-48590bf42ed5-kube-api-access-vrcnk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gl6jb\" (UID: \"2f9b62c3-bb42-4ed5-b734-48590bf42ed5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"
Jul 28 20:15:57.749722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:15:57.749626    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"
Jul 28 20:16:10.397119 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:10.397090    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"]
Jul 28 20:16:10.399318 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:16:10.399287    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f9b62c3_bb42_4ed5_b734_48590bf42ed5.slice/crio-9e36dc9b5ba6cd92a3ad3fdf8000ea9ce9d8ab133ee22bfbbea269591314de98 WatchSource:0}: Error finding container 9e36dc9b5ba6cd92a3ad3fdf8000ea9ce9d8ab133ee22bfbbea269591314de98: Status 404 returned error can't find the container with id 9e36dc9b5ba6cd92a3ad3fdf8000ea9ce9d8ab133ee22bfbbea269591314de98
Jul 28 20:16:10.585217 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:10.585109    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd" event={"ID":"7f9febd1-7e8a-40a9-829f-4151279e95dc","Type":"ContainerStarted","Data":"163f467267288ff6403836594c552d669f6614db86d47719e1c9a427deb8858e"}
Jul 28 20:16:10.586434 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:10.586409    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb" event={"ID":"2f9b62c3-bb42-4ed5-b734-48590bf42ed5","Type":"ContainerStarted","Data":"9e36dc9b5ba6cd92a3ad3fdf8000ea9ce9d8ab133ee22bfbbea269591314de98"}
Jul 28 20:16:10.606576 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:10.606503    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kzlwd" podStartSLOduration=1.615428687 podStartE2EDuration="24.606488493s" totalImagesPullingTime="22.991059806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:16:10.605014699 +0000 UTC m=+447.847107922" watchObservedRunningTime="2026-07-28 20:16:10.606488493 +0000 UTC m=+447.848581863"
Jul 28 20:16:15.614656 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:15.614616    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb" event={"ID":"2f9b62c3-bb42-4ed5-b734-48590bf42ed5","Type":"ContainerStarted","Data":"c126591a6d3b84efd3b8b12ef210821304a8a5d29909228c2c51ebff5e8bb28f"}
Jul 28 20:16:15.615107 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:15.614755    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"
Jul 28 20:16:15.635164 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:15.635101    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb" podStartSLOduration=13.840319979 podStartE2EDuration="18.635078209s" totalImagesPullingTime="4.79475823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:16:15.633861843 +0000 UTC m=+452.875955067" watchObservedRunningTime="2026-07-28 20:16:15.635078209 +0000 UTC m=+452.877171435"
Jul 28 20:16:26.621241 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:26.621206    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"
Jul 28 20:16:38.947958 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:38.947918    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"]
Jul 28 20:16:38.948400 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:38.948150    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb" podUID="2f9b62c3-bb42-4ed5-b734-48590bf42ed5" containerName="manager" containerID="cri-o://c126591a6d3b84efd3b8b12ef210821304a8a5d29909228c2c51ebff5e8bb28f" gracePeriod=10
Jul 28 20:16:39.207390 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.207301    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"
Jul 28 20:16:39.344183 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.344144    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrcnk\" (UniqueName: \"kubernetes.io/projected/2f9b62c3-bb42-4ed5-b734-48590bf42ed5-kube-api-access-vrcnk\") pod \"2f9b62c3-bb42-4ed5-b734-48590bf42ed5\" (UID: \"2f9b62c3-bb42-4ed5-b734-48590bf42ed5\") "
Jul 28 20:16:39.344183 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.344190    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f9b62c3-bb42-4ed5-b734-48590bf42ed5-extensions-socket-volume\") pod \"2f9b62c3-bb42-4ed5-b734-48590bf42ed5\" (UID: \"2f9b62c3-bb42-4ed5-b734-48590bf42ed5\") "
Jul 28 20:16:39.344760 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.344735    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f9b62c3-bb42-4ed5-b734-48590bf42ed5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2f9b62c3-bb42-4ed5-b734-48590bf42ed5" (UID: "2f9b62c3-bb42-4ed5-b734-48590bf42ed5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:16:39.346375 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.346355    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f9b62c3-bb42-4ed5-b734-48590bf42ed5-kube-api-access-vrcnk" (OuterVolumeSpecName: "kube-api-access-vrcnk") pod "2f9b62c3-bb42-4ed5-b734-48590bf42ed5" (UID: "2f9b62c3-bb42-4ed5-b734-48590bf42ed5"). InnerVolumeSpecName "kube-api-access-vrcnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:16:39.445694 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.445650    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrcnk\" (UniqueName: \"kubernetes.io/projected/2f9b62c3-bb42-4ed5-b734-48590bf42ed5-kube-api-access-vrcnk\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:16:39.445694 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.445684    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f9b62c3-bb42-4ed5-b734-48590bf42ed5-extensions-socket-volume\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:16:39.737053 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.737017    2558 generic.go:356] "Generic (PLEG): container finished" podID="2f9b62c3-bb42-4ed5-b734-48590bf42ed5" containerID="c126591a6d3b84efd3b8b12ef210821304a8a5d29909228c2c51ebff5e8bb28f" exitCode=0
Jul 28 20:16:39.737272 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.737065    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"
Jul 28 20:16:39.737272 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.737115    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb" event={"ID":"2f9b62c3-bb42-4ed5-b734-48590bf42ed5","Type":"ContainerDied","Data":"c126591a6d3b84efd3b8b12ef210821304a8a5d29909228c2c51ebff5e8bb28f"}
Jul 28 20:16:39.737272 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.737170    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb" event={"ID":"2f9b62c3-bb42-4ed5-b734-48590bf42ed5","Type":"ContainerDied","Data":"9e36dc9b5ba6cd92a3ad3fdf8000ea9ce9d8ab133ee22bfbbea269591314de98"}
Jul 28 20:16:39.737272 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.737200    2558 scope.go:117] "RemoveContainer" containerID="c126591a6d3b84efd3b8b12ef210821304a8a5d29909228c2c51ebff5e8bb28f"
Jul 28 20:16:39.747885 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.747861    2558 scope.go:117] "RemoveContainer" containerID="c126591a6d3b84efd3b8b12ef210821304a8a5d29909228c2c51ebff5e8bb28f"
Jul 28 20:16:39.748203 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:16:39.748167    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c126591a6d3b84efd3b8b12ef210821304a8a5d29909228c2c51ebff5e8bb28f\": container with ID starting with c126591a6d3b84efd3b8b12ef210821304a8a5d29909228c2c51ebff5e8bb28f not found: ID does not exist" containerID="c126591a6d3b84efd3b8b12ef210821304a8a5d29909228c2c51ebff5e8bb28f"
Jul 28 20:16:39.748290 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.748213    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c126591a6d3b84efd3b8b12ef210821304a8a5d29909228c2c51ebff5e8bb28f"} err="failed to get container status \"c126591a6d3b84efd3b8b12ef210821304a8a5d29909228c2c51ebff5e8bb28f\": rpc error: code = NotFound desc = could not find container \"c126591a6d3b84efd3b8b12ef210821304a8a5d29909228c2c51ebff5e8bb28f\": container with ID starting with c126591a6d3b84efd3b8b12ef210821304a8a5d29909228c2c51ebff5e8bb28f not found: ID does not exist"
Jul 28 20:16:39.769798 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.769764    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"]
Jul 28 20:16:39.774136 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:39.774107    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gl6jb"]
Jul 28 20:16:41.340444 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:41.340404    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f9b62c3-bb42-4ed5-b734-48590bf42ed5" path="/var/lib/kubelet/pods/2f9b62c3-bb42-4ed5-b734-48590bf42ed5/volumes"
Jul 28 20:16:45.958587 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:45.958536    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:16:45.959191 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:45.959170    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f9b62c3-bb42-4ed5-b734-48590bf42ed5" containerName="manager"
Jul 28 20:16:45.959278 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:45.959194    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f9b62c3-bb42-4ed5-b734-48590bf42ed5" containerName="manager"
Jul 28 20:16:45.959331 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:45.959300    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f9b62c3-bb42-4ed5-b734-48590bf42ed5" containerName="manager"
Jul 28 20:16:46.017227 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.017186    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:16:46.017227 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.017234    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:16:46.017485 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.017333    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"
Jul 28 20:16:46.019838 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.019809    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 20:16:46.107041 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.107003    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txsgs\" (UniqueName: \"kubernetes.io/projected/a17fc07b-65ac-4da7-b9d5-1dc67bb60f10-kube-api-access-txsgs\") pod \"limitador-limitador-54d6d988d5-tpzd2\" (UID: \"a17fc07b-65ac-4da7-b9d5-1dc67bb60f10\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"
Jul 28 20:16:46.107243 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.107071    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a17fc07b-65ac-4da7-b9d5-1dc67bb60f10-config-file\") pod \"limitador-limitador-54d6d988d5-tpzd2\" (UID: \"a17fc07b-65ac-4da7-b9d5-1dc67bb60f10\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"
Jul 28 20:16:46.208076 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.208033    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a17fc07b-65ac-4da7-b9d5-1dc67bb60f10-config-file\") pod \"limitador-limitador-54d6d988d5-tpzd2\" (UID: \"a17fc07b-65ac-4da7-b9d5-1dc67bb60f10\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"
Jul 28 20:16:46.208273 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.208142    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txsgs\" (UniqueName: \"kubernetes.io/projected/a17fc07b-65ac-4da7-b9d5-1dc67bb60f10-kube-api-access-txsgs\") pod \"limitador-limitador-54d6d988d5-tpzd2\" (UID: \"a17fc07b-65ac-4da7-b9d5-1dc67bb60f10\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"
Jul 28 20:16:46.208785 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.208725    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a17fc07b-65ac-4da7-b9d5-1dc67bb60f10-config-file\") pod \"limitador-limitador-54d6d988d5-tpzd2\" (UID: \"a17fc07b-65ac-4da7-b9d5-1dc67bb60f10\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"
Jul 28 20:16:46.216414 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.216380    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txsgs\" (UniqueName: \"kubernetes.io/projected/a17fc07b-65ac-4da7-b9d5-1dc67bb60f10-kube-api-access-txsgs\") pod \"limitador-limitador-54d6d988d5-tpzd2\" (UID: \"a17fc07b-65ac-4da7-b9d5-1dc67bb60f10\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"
Jul 28 20:16:46.327615 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.327542    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"
Jul 28 20:16:46.463304 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.463227    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:16:46.466784 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:16:46.466758    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda17fc07b_65ac_4da7_b9d5_1dc67bb60f10.slice/crio-8525362698e49d281bb30426304f8438c4f85d305128e75ca99281afe6ee234c WatchSource:0}: Error finding container 8525362698e49d281bb30426304f8438c4f85d305128e75ca99281afe6ee234c: Status 404 returned error can't find the container with id 8525362698e49d281bb30426304f8438c4f85d305128e75ca99281afe6ee234c
Jul 28 20:16:46.549485 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.549443    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-bjbkg"]
Jul 28 20:16:46.553909 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.553891    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bjbkg"
Jul 28 20:16:46.556175 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.556156    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-llp4v\""
Jul 28 20:16:46.560586 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.560541    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bjbkg"]
Jul 28 20:16:46.711904 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.711864    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nt9z\" (UniqueName: \"kubernetes.io/projected/e621809c-c1a7-4537-8c0b-656ae5e87907-kube-api-access-9nt9z\") pod \"authorino-75d556bd77-bjbkg\" (UID: \"e621809c-c1a7-4537-8c0b-656ae5e87907\") " pod="kuadrant-system/authorino-75d556bd77-bjbkg"
Jul 28 20:16:46.771861 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.771827    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpzd2" event={"ID":"a17fc07b-65ac-4da7-b9d5-1dc67bb60f10","Type":"ContainerStarted","Data":"8525362698e49d281bb30426304f8438c4f85d305128e75ca99281afe6ee234c"}
Jul 28 20:16:46.812502 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.812460    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nt9z\" (UniqueName: \"kubernetes.io/projected/e621809c-c1a7-4537-8c0b-656ae5e87907-kube-api-access-9nt9z\") pod \"authorino-75d556bd77-bjbkg\" (UID: \"e621809c-c1a7-4537-8c0b-656ae5e87907\") " pod="kuadrant-system/authorino-75d556bd77-bjbkg"
Jul 28 20:16:46.821658 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.821624    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nt9z\" (UniqueName: \"kubernetes.io/projected/e621809c-c1a7-4537-8c0b-656ae5e87907-kube-api-access-9nt9z\") pod \"authorino-75d556bd77-bjbkg\" (UID: \"e621809c-c1a7-4537-8c0b-656ae5e87907\") " pod="kuadrant-system/authorino-75d556bd77-bjbkg"
Jul 28 20:16:46.864928 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:46.864893    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bjbkg"
Jul 28 20:16:47.060350 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:47.060310    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bjbkg"]
Jul 28 20:16:47.063646 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:16:47.063612    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode621809c_c1a7_4537_8c0b_656ae5e87907.slice/crio-135da52cc3e806df4c1990bdfeb2eecfc04e201eb3d29e3a2f428a9ffecbbc3d WatchSource:0}: Error finding container 135da52cc3e806df4c1990bdfeb2eecfc04e201eb3d29e3a2f428a9ffecbbc3d: Status 404 returned error can't find the container with id 135da52cc3e806df4c1990bdfeb2eecfc04e201eb3d29e3a2f428a9ffecbbc3d
Jul 28 20:16:47.780617 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:47.780532    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bjbkg" event={"ID":"e621809c-c1a7-4537-8c0b-656ae5e87907","Type":"ContainerStarted","Data":"135da52cc3e806df4c1990bdfeb2eecfc04e201eb3d29e3a2f428a9ffecbbc3d"}
Jul 28 20:16:51.806166 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:51.806126    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpzd2" event={"ID":"a17fc07b-65ac-4da7-b9d5-1dc67bb60f10","Type":"ContainerStarted","Data":"375b22c400000442fa291bee0b8aeb9a6dfac05d37c52f76b473b81276c8e547"}
Jul 28 20:16:51.806662 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:51.806198    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"
Jul 28 20:16:51.807622 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:51.807592    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bjbkg" event={"ID":"e621809c-c1a7-4537-8c0b-656ae5e87907","Type":"ContainerStarted","Data":"ed54d231555ac4c55eeffa7aa17098ae0ceb4ebe4b1d6d036efd59c3f35a3a19"}
Jul 28 20:16:51.820870 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:51.820812    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpzd2" podStartSLOduration=2.373600416 podStartE2EDuration="6.820792275s" totalImagesPullingTime="4.447191859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:16:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:16:51.820473999 +0000 UTC m=+489.062567224" watchObservedRunningTime="2026-07-28 20:16:51.820792275 +0000 UTC m=+489.062885498"
Jul 28 20:16:51.833902 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:16:51.833847    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-bjbkg" podStartSLOduration=1.972756565 podStartE2EDuration="5.833832343s" totalImagesPullingTime="3.861075778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:16:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:16:51.833328926 +0000 UTC m=+489.075422151" watchObservedRunningTime="2026-07-28 20:16:51.833832343 +0000 UTC m=+489.075925615"
Jul 28 20:17:02.814782 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:02.814746    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"
Jul 28 20:17:26.295941 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:26.295904    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7454679f7-xbm5g"]
Jul 28 20:17:26.302684 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:26.302655    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7454679f7-xbm5g"
Jul 28 20:17:26.305135 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:26.305109    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 20:17:26.305932 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:26.305909    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7454679f7-xbm5g"]
Jul 28 20:17:26.379171 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:26.379132    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1e643e99-97b2-43bb-bba8-917839137789-tls-cert\") pod \"authorino-7454679f7-xbm5g\" (UID: \"1e643e99-97b2-43bb-bba8-917839137789\") " pod="kuadrant-system/authorino-7454679f7-xbm5g"
Jul 28 20:17:26.379361 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:26.379202    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66d2k\" (UniqueName: \"kubernetes.io/projected/1e643e99-97b2-43bb-bba8-917839137789-kube-api-access-66d2k\") pod \"authorino-7454679f7-xbm5g\" (UID: \"1e643e99-97b2-43bb-bba8-917839137789\") " pod="kuadrant-system/authorino-7454679f7-xbm5g"
Jul 28 20:17:26.480179 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:26.480140    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1e643e99-97b2-43bb-bba8-917839137789-tls-cert\") pod \"authorino-7454679f7-xbm5g\" (UID: \"1e643e99-97b2-43bb-bba8-917839137789\") " pod="kuadrant-system/authorino-7454679f7-xbm5g"
Jul 28 20:17:26.480509 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:26.480487    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66d2k\" (UniqueName: \"kubernetes.io/projected/1e643e99-97b2-43bb-bba8-917839137789-kube-api-access-66d2k\") pod \"authorino-7454679f7-xbm5g\" (UID: \"1e643e99-97b2-43bb-bba8-917839137789\") " pod="kuadrant-system/authorino-7454679f7-xbm5g"
Jul 28 20:17:26.482872 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:26.482837    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1e643e99-97b2-43bb-bba8-917839137789-tls-cert\") pod \"authorino-7454679f7-xbm5g\" (UID: \"1e643e99-97b2-43bb-bba8-917839137789\") " pod="kuadrant-system/authorino-7454679f7-xbm5g"
Jul 28 20:17:26.488162 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:26.488128    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66d2k\" (UniqueName: \"kubernetes.io/projected/1e643e99-97b2-43bb-bba8-917839137789-kube-api-access-66d2k\") pod \"authorino-7454679f7-xbm5g\" (UID: \"1e643e99-97b2-43bb-bba8-917839137789\") " pod="kuadrant-system/authorino-7454679f7-xbm5g"
Jul 28 20:17:26.612932 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:26.612839    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7454679f7-xbm5g"
Jul 28 20:17:26.748017 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:26.747929    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7454679f7-xbm5g"]
Jul 28 20:17:26.750416 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:17:26.750385    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e643e99_97b2_43bb_bba8_917839137789.slice/crio-d0050c2207c7b63b2d6fe93317d03fbb3ea84e4706f26a174398f0829a26dbbb WatchSource:0}: Error finding container d0050c2207c7b63b2d6fe93317d03fbb3ea84e4706f26a174398f0829a26dbbb: Status 404 returned error can't find the container with id d0050c2207c7b63b2d6fe93317d03fbb3ea84e4706f26a174398f0829a26dbbb
Jul 28 20:17:26.984833 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:26.984740    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7454679f7-xbm5g" event={"ID":"1e643e99-97b2-43bb-bba8-917839137789","Type":"ContainerStarted","Data":"d0050c2207c7b63b2d6fe93317d03fbb3ea84e4706f26a174398f0829a26dbbb"}
Jul 28 20:17:27.991307 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:27.991270    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7454679f7-xbm5g" event={"ID":"1e643e99-97b2-43bb-bba8-917839137789","Type":"ContainerStarted","Data":"0a13160dbe913c635fc85d83d7e9acd458f5929bc6a3e8b209d962cc5f3e7866"}
Jul 28 20:17:28.005982 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:28.005938    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7454679f7-xbm5g" podStartSLOduration=1.5213719970000001 podStartE2EDuration="2.005918124s" totalImagesPullingTime="484.546127ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:17:28.004811625 +0000 UTC m=+525.246904849" watchObservedRunningTime="2026-07-28 20:17:28.005918124 +0000 UTC m=+525.248011346"
Jul 28 20:17:28.029421 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:28.029384    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bjbkg"]
Jul 28 20:17:28.029626 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:28.029600    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-bjbkg" podUID="e621809c-c1a7-4537-8c0b-656ae5e87907" containerName="authorino" containerID="cri-o://ed54d231555ac4c55eeffa7aa17098ae0ceb4ebe4b1d6d036efd59c3f35a3a19" gracePeriod=30
Jul 28 20:17:28.282199 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:28.282170    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bjbkg"
Jul 28 20:17:28.397752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:28.397717    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nt9z\" (UniqueName: \"kubernetes.io/projected/e621809c-c1a7-4537-8c0b-656ae5e87907-kube-api-access-9nt9z\") pod \"e621809c-c1a7-4537-8c0b-656ae5e87907\" (UID: \"e621809c-c1a7-4537-8c0b-656ae5e87907\") "
Jul 28 20:17:28.399917 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:28.399885    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e621809c-c1a7-4537-8c0b-656ae5e87907-kube-api-access-9nt9z" (OuterVolumeSpecName: "kube-api-access-9nt9z") pod "e621809c-c1a7-4537-8c0b-656ae5e87907" (UID: "e621809c-c1a7-4537-8c0b-656ae5e87907"). InnerVolumeSpecName "kube-api-access-9nt9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:17:28.499130 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:28.499084    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nt9z\" (UniqueName: \"kubernetes.io/projected/e621809c-c1a7-4537-8c0b-656ae5e87907-kube-api-access-9nt9z\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:17:28.998249 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:28.998209    2558 generic.go:356] "Generic (PLEG): container finished" podID="e621809c-c1a7-4537-8c0b-656ae5e87907" containerID="ed54d231555ac4c55eeffa7aa17098ae0ceb4ebe4b1d6d036efd59c3f35a3a19" exitCode=0
Jul 28 20:17:28.998732 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:28.998257    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bjbkg"
Jul 28 20:17:28.998732 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:28.998295    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bjbkg" event={"ID":"e621809c-c1a7-4537-8c0b-656ae5e87907","Type":"ContainerDied","Data":"ed54d231555ac4c55eeffa7aa17098ae0ceb4ebe4b1d6d036efd59c3f35a3a19"}
Jul 28 20:17:28.998732 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:28.998342    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bjbkg" event={"ID":"e621809c-c1a7-4537-8c0b-656ae5e87907","Type":"ContainerDied","Data":"135da52cc3e806df4c1990bdfeb2eecfc04e201eb3d29e3a2f428a9ffecbbc3d"}
Jul 28 20:17:28.998732 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:28.998363    2558 scope.go:117] "RemoveContainer" containerID="ed54d231555ac4c55eeffa7aa17098ae0ceb4ebe4b1d6d036efd59c3f35a3a19"
Jul 28 20:17:29.007320 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:29.007301    2558 scope.go:117] "RemoveContainer" containerID="ed54d231555ac4c55eeffa7aa17098ae0ceb4ebe4b1d6d036efd59c3f35a3a19"
Jul 28 20:17:29.007632 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:17:29.007609    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed54d231555ac4c55eeffa7aa17098ae0ceb4ebe4b1d6d036efd59c3f35a3a19\": container with ID starting with ed54d231555ac4c55eeffa7aa17098ae0ceb4ebe4b1d6d036efd59c3f35a3a19 not found: ID does not exist" containerID="ed54d231555ac4c55eeffa7aa17098ae0ceb4ebe4b1d6d036efd59c3f35a3a19"
Jul 28 20:17:29.007711 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:29.007645    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed54d231555ac4c55eeffa7aa17098ae0ceb4ebe4b1d6d036efd59c3f35a3a19"} err="failed to get container status \"ed54d231555ac4c55eeffa7aa17098ae0ceb4ebe4b1d6d036efd59c3f35a3a19\": rpc error: code = NotFound desc = could not find container \"ed54d231555ac4c55eeffa7aa17098ae0ceb4ebe4b1d6d036efd59c3f35a3a19\": container with ID starting with ed54d231555ac4c55eeffa7aa17098ae0ceb4ebe4b1d6d036efd59c3f35a3a19 not found: ID does not exist"
Jul 28 20:17:29.018961 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:29.018932    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bjbkg"]
Jul 28 20:17:29.020567 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:29.020537    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bjbkg"]
Jul 28 20:17:29.340716 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:17:29.340685    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e621809c-c1a7-4537-8c0b-656ae5e87907" path="/var/lib/kubelet/pods/e621809c-c1a7-4537-8c0b-656ae5e87907/volumes"
Jul 28 20:18:11.609187 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:11.609146    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:18:29.591857 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:29.591807    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:18:43.174692 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:43.174658    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:18:43.177166 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:43.177141    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:18:45.718221 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:45.718182    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:18:51.994658 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:51.994618    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"]
Jul 28 20:18:51.996978 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:51.995015    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e621809c-c1a7-4537-8c0b-656ae5e87907" containerName="authorino"
Jul 28 20:18:51.996978 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:51.995028    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e621809c-c1a7-4537-8c0b-656ae5e87907" containerName="authorino"
Jul 28 20:18:51.996978 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:51.995111    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e621809c-c1a7-4537-8c0b-656ae5e87907" containerName="authorino"
Jul 28 20:18:51.998055 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:51.998032    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.000464 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.000440    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 20:18:52.000607 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.000588    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 20:18:52.000668 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.000651    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xw4fk\""
Jul 28 20:18:52.001626 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.001607    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 28 20:18:52.016925 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.016856    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"]
Jul 28 20:18:52.093497 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.093451    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fba53f0-8999-4d98-bb12-5d3e13cf657c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.093497 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.093496    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pldpq\" (UniqueName: \"kubernetes.io/projected/2fba53f0-8999-4d98-bb12-5d3e13cf657c-kube-api-access-pldpq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.093752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.093590    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fba53f0-8999-4d98-bb12-5d3e13cf657c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.093752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.093656    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fba53f0-8999-4d98-bb12-5d3e13cf657c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.093752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.093700    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fba53f0-8999-4d98-bb12-5d3e13cf657c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.093752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.093746    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fba53f0-8999-4d98-bb12-5d3e13cf657c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.194785 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.194741    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fba53f0-8999-4d98-bb12-5d3e13cf657c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.194785 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.194786    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pldpq\" (UniqueName: \"kubernetes.io/projected/2fba53f0-8999-4d98-bb12-5d3e13cf657c-kube-api-access-pldpq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.195014 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.194825    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fba53f0-8999-4d98-bb12-5d3e13cf657c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.195014 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.194867    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fba53f0-8999-4d98-bb12-5d3e13cf657c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.195110 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.195019    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fba53f0-8999-4d98-bb12-5d3e13cf657c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.195110 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.195077    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fba53f0-8999-4d98-bb12-5d3e13cf657c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.195235 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.195215    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fba53f0-8999-4d98-bb12-5d3e13cf657c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.195331 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.195308    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fba53f0-8999-4d98-bb12-5d3e13cf657c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.195401 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.195378    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fba53f0-8999-4d98-bb12-5d3e13cf657c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.197117 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.197099    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fba53f0-8999-4d98-bb12-5d3e13cf657c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.197355 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.197337    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fba53f0-8999-4d98-bb12-5d3e13cf657c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.203905 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.203883    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pldpq\" (UniqueName: \"kubernetes.io/projected/2fba53f0-8999-4d98-bb12-5d3e13cf657c-kube-api-access-pldpq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r2xzn\" (UID: \"2fba53f0-8999-4d98-bb12-5d3e13cf657c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.309263 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.309229    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:18:52.453015 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.452986    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"]
Jul 28 20:18:52.455415 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:18:52.455388    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fba53f0_8999_4d98_bb12_5d3e13cf657c.slice/crio-98f871e16bb95ed744e921dedda57422d71a4912cf88983acad38d47b9ad7d57 WatchSource:0}: Error finding container 98f871e16bb95ed744e921dedda57422d71a4912cf88983acad38d47b9ad7d57: Status 404 returned error can't find the container with id 98f871e16bb95ed744e921dedda57422d71a4912cf88983acad38d47b9ad7d57
Jul 28 20:18:52.457116 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:52.457096    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:18:53.188830 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:53.188787    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:18:53.414616 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:53.414573    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn" event={"ID":"2fba53f0-8999-4d98-bb12-5d3e13cf657c","Type":"ContainerStarted","Data":"98f871e16bb95ed744e921dedda57422d71a4912cf88983acad38d47b9ad7d57"}
Jul 28 20:18:54.421720 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:54.421681    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn" event={"ID":"2fba53f0-8999-4d98-bb12-5d3e13cf657c","Type":"ContainerStarted","Data":"24870d65c578936273e29a1c0bf9bceff8d1bdd130c7bfc8fa08420caa915853"}
Jul 28 20:18:54.445029 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:18:54.444967    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn" podStartSLOduration=2.264976421 podStartE2EDuration="3.444950447s" totalImagesPullingTime="1.179974026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:18:54.443010551 +0000 UTC m=+611.685103778" watchObservedRunningTime="2026-07-28 20:18:54.444950447 +0000 UTC m=+611.687043670"
Jul 28 20:19:02.182014 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:02.181977    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:19:02.310098 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:02.310060    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:19:02.310436 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:02.310391    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:19:02.326181 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:02.326145    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:19:02.477409 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:02.477320    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r2xzn"
Jul 28 20:19:17.287360 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:17.287256    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:19:18.614577 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.614161    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"]
Jul 28 20:19:18.622593 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.621712    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.629301 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.629266    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 20:19:18.638171 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.638123    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"]
Jul 28 20:19:18.750857 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.750814    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ec4a624-13bd-447d-8f96-f1c8c7030d29-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.751090 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.750889    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ec4a624-13bd-447d-8f96-f1c8c7030d29-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.751090 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.750965    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ec4a624-13bd-447d-8f96-f1c8c7030d29-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.751090 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.750999    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ec4a624-13bd-447d-8f96-f1c8c7030d29-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.751090 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.751033    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x9rw\" (UniqueName: \"kubernetes.io/projected/5ec4a624-13bd-447d-8f96-f1c8c7030d29-kube-api-access-5x9rw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.751090 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.751073    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ec4a624-13bd-447d-8f96-f1c8c7030d29-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.851921 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.851875    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ec4a624-13bd-447d-8f96-f1c8c7030d29-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.851921 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.851918    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ec4a624-13bd-447d-8f96-f1c8c7030d29-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.852175 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.852021    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x9rw\" (UniqueName: \"kubernetes.io/projected/5ec4a624-13bd-447d-8f96-f1c8c7030d29-kube-api-access-5x9rw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.852175 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.852085    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ec4a624-13bd-447d-8f96-f1c8c7030d29-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.852296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.852175    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ec4a624-13bd-447d-8f96-f1c8c7030d29-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.852296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.852265    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ec4a624-13bd-447d-8f96-f1c8c7030d29-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.852296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.852288    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ec4a624-13bd-447d-8f96-f1c8c7030d29-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.852542 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.852517    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ec4a624-13bd-447d-8f96-f1c8c7030d29-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.852641 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.852626    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ec4a624-13bd-447d-8f96-f1c8c7030d29-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.854229 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.854201    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ec4a624-13bd-447d-8f96-f1c8c7030d29-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.854583 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.854544    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ec4a624-13bd-447d-8f96-f1c8c7030d29-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.865908 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.865830    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x9rw\" (UniqueName: \"kubernetes.io/projected/5ec4a624-13bd-447d-8f96-f1c8c7030d29-kube-api-access-5x9rw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb\" (UID: \"5ec4a624-13bd-447d-8f96-f1c8c7030d29\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:18.947875 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:18.947830    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:19.096856 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:19.096828    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"]
Jul 28 20:19:19.099700 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:19:19.099668    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ec4a624_13bd_447d_8f96_f1c8c7030d29.slice/crio-177e8241fabe18908f5d8999e307ff147371d533d0b68bd1644d4feb7cb0d19f WatchSource:0}: Error finding container 177e8241fabe18908f5d8999e307ff147371d533d0b68bd1644d4feb7cb0d19f: Status 404 returned error can't find the container with id 177e8241fabe18908f5d8999e307ff147371d533d0b68bd1644d4feb7cb0d19f
Jul 28 20:19:19.549276 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:19.549229    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb" event={"ID":"5ec4a624-13bd-447d-8f96-f1c8c7030d29","Type":"ContainerStarted","Data":"932894c497851512ba022f85d0ea949ce63274f0209a2736fb8a2cd90e30b2bb"}
Jul 28 20:19:19.549276 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:19.549280    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb" event={"ID":"5ec4a624-13bd-447d-8f96-f1c8c7030d29","Type":"ContainerStarted","Data":"177e8241fabe18908f5d8999e307ff147371d533d0b68bd1644d4feb7cb0d19f"}
Jul 28 20:19:19.572768 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:19.572714    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb" podStartSLOduration=1.308252679 podStartE2EDuration="1.572697263s" totalImagesPullingTime="264.444584ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:19:19.569714727 +0000 UTC m=+636.811808005" watchObservedRunningTime="2026-07-28 20:19:19.572697263 +0000 UTC m=+636.814790486"
Jul 28 20:19:28.788282 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:28.788248    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:19:28.948205 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:28.948165    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:28.948205 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:28.948213    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:28.960710 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:28.960681    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:19:29.611543 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:19:29.611509    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb"
Jul 28 20:20:40.034088 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.034004    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c987cb57b-gcgp7"]
Jul 28 20:20:40.036440 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.036421    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c987cb57b-gcgp7"
Jul 28 20:20:40.047725 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.047697    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c987cb57b-gcgp7"]
Jul 28 20:20:40.144689 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.144644    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b72fb21e-3843-49e2-af39-d038022fa1e7-tls-cert\") pod \"authorino-7c987cb57b-gcgp7\" (UID: \"b72fb21e-3843-49e2-af39-d038022fa1e7\") " pod="kuadrant-system/authorino-7c987cb57b-gcgp7"
Jul 28 20:20:40.144689 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.144686    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb2sr\" (UniqueName: \"kubernetes.io/projected/b72fb21e-3843-49e2-af39-d038022fa1e7-kube-api-access-bb2sr\") pod \"authorino-7c987cb57b-gcgp7\" (UID: \"b72fb21e-3843-49e2-af39-d038022fa1e7\") " pod="kuadrant-system/authorino-7c987cb57b-gcgp7"
Jul 28 20:20:40.245767 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.245710    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b72fb21e-3843-49e2-af39-d038022fa1e7-tls-cert\") pod \"authorino-7c987cb57b-gcgp7\" (UID: \"b72fb21e-3843-49e2-af39-d038022fa1e7\") " pod="kuadrant-system/authorino-7c987cb57b-gcgp7"
Jul 28 20:20:40.245767 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.245773    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb2sr\" (UniqueName: \"kubernetes.io/projected/b72fb21e-3843-49e2-af39-d038022fa1e7-kube-api-access-bb2sr\") pod \"authorino-7c987cb57b-gcgp7\" (UID: \"b72fb21e-3843-49e2-af39-d038022fa1e7\") " pod="kuadrant-system/authorino-7c987cb57b-gcgp7"
Jul 28 20:20:40.248141 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.248120    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b72fb21e-3843-49e2-af39-d038022fa1e7-tls-cert\") pod \"authorino-7c987cb57b-gcgp7\" (UID: \"b72fb21e-3843-49e2-af39-d038022fa1e7\") " pod="kuadrant-system/authorino-7c987cb57b-gcgp7"
Jul 28 20:20:40.254005 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.253975    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb2sr\" (UniqueName: \"kubernetes.io/projected/b72fb21e-3843-49e2-af39-d038022fa1e7-kube-api-access-bb2sr\") pod \"authorino-7c987cb57b-gcgp7\" (UID: \"b72fb21e-3843-49e2-af39-d038022fa1e7\") " pod="kuadrant-system/authorino-7c987cb57b-gcgp7"
Jul 28 20:20:40.346641 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.346536    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c987cb57b-gcgp7"
Jul 28 20:20:40.481547 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.481522    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c987cb57b-gcgp7"]
Jul 28 20:20:40.483688 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:20:40.483649    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb72fb21e_3843_49e2_af39_d038022fa1e7.slice/crio-a068a0afc0f1ce71110d4a4ad5e2982e91efca33cdb3ba506f7d9b37cce71bef WatchSource:0}: Error finding container a068a0afc0f1ce71110d4a4ad5e2982e91efca33cdb3ba506f7d9b37cce71bef: Status 404 returned error can't find the container with id a068a0afc0f1ce71110d4a4ad5e2982e91efca33cdb3ba506f7d9b37cce71bef
Jul 28 20:20:40.960253 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.960218    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c987cb57b-gcgp7" event={"ID":"b72fb21e-3843-49e2-af39-d038022fa1e7","Type":"ContainerStarted","Data":"25b299f89b10b5bd9ea04a32d7f21e4f81199063a6afa52b202d6d25c551d21c"}
Jul 28 20:20:40.960253 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.960258    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c987cb57b-gcgp7" event={"ID":"b72fb21e-3843-49e2-af39-d038022fa1e7","Type":"ContainerStarted","Data":"a068a0afc0f1ce71110d4a4ad5e2982e91efca33cdb3ba506f7d9b37cce71bef"}
Jul 28 20:20:40.975728 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:40.975676    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7c987cb57b-gcgp7" podStartSLOduration=0.575991575 podStartE2EDuration="975.655786ms" totalImagesPullingTime="399.664211ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:20:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:20:40.975192454 +0000 UTC m=+718.217285756" watchObservedRunningTime="2026-07-28 20:20:40.975655786 +0000 UTC m=+718.217749010"
Jul 28 20:20:41.000050 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.000016    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7454679f7-xbm5g"]
Jul 28 20:20:41.000549 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.000227    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7454679f7-xbm5g" podUID="1e643e99-97b2-43bb-bba8-917839137789" containerName="authorino" containerID="cri-o://0a13160dbe913c635fc85d83d7e9acd458f5929bc6a3e8b209d962cc5f3e7866" gracePeriod=30
Jul 28 20:20:41.247748 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.247716    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7454679f7-xbm5g"
Jul 28 20:20:41.356913 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.356882    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1e643e99-97b2-43bb-bba8-917839137789-tls-cert\") pod \"1e643e99-97b2-43bb-bba8-917839137789\" (UID: \"1e643e99-97b2-43bb-bba8-917839137789\") "
Jul 28 20:20:41.356913 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.356918    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66d2k\" (UniqueName: \"kubernetes.io/projected/1e643e99-97b2-43bb-bba8-917839137789-kube-api-access-66d2k\") pod \"1e643e99-97b2-43bb-bba8-917839137789\" (UID: \"1e643e99-97b2-43bb-bba8-917839137789\") "
Jul 28 20:20:41.359098 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.359058    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e643e99-97b2-43bb-bba8-917839137789-kube-api-access-66d2k" (OuterVolumeSpecName: "kube-api-access-66d2k") pod "1e643e99-97b2-43bb-bba8-917839137789" (UID: "1e643e99-97b2-43bb-bba8-917839137789"). InnerVolumeSpecName "kube-api-access-66d2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:20:41.369236 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.369188    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e643e99-97b2-43bb-bba8-917839137789-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1e643e99-97b2-43bb-bba8-917839137789" (UID: "1e643e99-97b2-43bb-bba8-917839137789"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:20:41.458021 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.457917    2558 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1e643e99-97b2-43bb-bba8-917839137789-tls-cert\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:20:41.458021 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.457966    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66d2k\" (UniqueName: \"kubernetes.io/projected/1e643e99-97b2-43bb-bba8-917839137789-kube-api-access-66d2k\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:20:41.967527 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.967492    2558 generic.go:356] "Generic (PLEG): container finished" podID="1e643e99-97b2-43bb-bba8-917839137789" containerID="0a13160dbe913c635fc85d83d7e9acd458f5929bc6a3e8b209d962cc5f3e7866" exitCode=0
Jul 28 20:20:41.967732 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.967551    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7454679f7-xbm5g"
Jul 28 20:20:41.967732 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.967583    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7454679f7-xbm5g" event={"ID":"1e643e99-97b2-43bb-bba8-917839137789","Type":"ContainerDied","Data":"0a13160dbe913c635fc85d83d7e9acd458f5929bc6a3e8b209d962cc5f3e7866"}
Jul 28 20:20:41.967732 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.967625    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7454679f7-xbm5g" event={"ID":"1e643e99-97b2-43bb-bba8-917839137789","Type":"ContainerDied","Data":"d0050c2207c7b63b2d6fe93317d03fbb3ea84e4706f26a174398f0829a26dbbb"}
Jul 28 20:20:41.967732 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.967641    2558 scope.go:117] "RemoveContainer" containerID="0a13160dbe913c635fc85d83d7e9acd458f5929bc6a3e8b209d962cc5f3e7866"
Jul 28 20:20:41.977900 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.977873    2558 scope.go:117] "RemoveContainer" containerID="0a13160dbe913c635fc85d83d7e9acd458f5929bc6a3e8b209d962cc5f3e7866"
Jul 28 20:20:41.978270 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:20:41.978241    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a13160dbe913c635fc85d83d7e9acd458f5929bc6a3e8b209d962cc5f3e7866\": container with ID starting with 0a13160dbe913c635fc85d83d7e9acd458f5929bc6a3e8b209d962cc5f3e7866 not found: ID does not exist" containerID="0a13160dbe913c635fc85d83d7e9acd458f5929bc6a3e8b209d962cc5f3e7866"
Jul 28 20:20:41.978388 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.978279    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a13160dbe913c635fc85d83d7e9acd458f5929bc6a3e8b209d962cc5f3e7866"} err="failed to get container status \"0a13160dbe913c635fc85d83d7e9acd458f5929bc6a3e8b209d962cc5f3e7866\": rpc error: code = NotFound desc = could not find container \"0a13160dbe913c635fc85d83d7e9acd458f5929bc6a3e8b209d962cc5f3e7866\": container with ID starting with 0a13160dbe913c635fc85d83d7e9acd458f5929bc6a3e8b209d962cc5f3e7866 not found: ID does not exist"
Jul 28 20:20:41.993135 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.993093    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7454679f7-xbm5g"]
Jul 28 20:20:41.997043 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:41.997009    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7454679f7-xbm5g"]
Jul 28 20:20:43.341101 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:20:43.341065    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e643e99-97b2-43bb-bba8-917839137789" path="/var/lib/kubelet/pods/1e643e99-97b2-43bb-bba8-917839137789/volumes"
Jul 28 20:21:08.592938 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:21:08.592900    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:21:20.384597 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:21:20.384541    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:21:28.726078 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:21:28.726033    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:21:40.987279 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:21:40.987238    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:23:17.820701 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:23:17.820662    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:23:28.123910 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:23:28.123872    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:23:43.217422 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:23:43.217349    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:23:43.217422 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:23:43.217393    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:24:03.420124 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:24:03.420082    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:24:12.507500 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:24:12.507462    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:24:22.888826 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:24:22.888788    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:24:31.107878 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:24:31.107832    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:24:42.336190 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:24:42.336152    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:24:52.202237 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:24:52.202204    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:25:02.996134 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:25:02.996095    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:25:12.884935 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:25:12.884844    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:25:19.091273 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:25:19.091234    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:25:26.988011 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:25:26.987964    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:25:38.501262 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:25:38.501218    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:25:50.298762 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:25:50.298726    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:26:15.596571 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:26:15.596515    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:26:26.597146 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:26:26.597104    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:26:37.684502 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:26:37.684407    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:26:53.404879 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:26:53.404837    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:27:02.781940 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:27:02.781898    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:27:12.396105 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:27:12.396068    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:27:21.922758 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:27:21.922718    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:27:31.189155 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:27:31.189114    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:27:41.392026 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:27:41.391983    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:27:51.496154 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:27:51.496102    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:28:43.252262 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:28:43.252231    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:28:43.254048 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:28:43.254020    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:29:32.900690 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:29:32.900647    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:29:43.490828 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:29:43.490737    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:29:58.785153 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:29:58.785114    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:30:09.004488 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:30:09.004447    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:30:21.204937 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:30:21.204900    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:30:29.422178 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:30:29.416596    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:30:39.796311 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:30:39.796268    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:30:50.608090 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:30:50.608045    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:31:00.008095 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:31:00.008055    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:31:10.998545 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:31:10.998458    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:31:22.409289 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:31:22.409250    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:31:29.092386 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:31:29.092341    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:31:50.306098 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:31:50.306058    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:32:00.597679 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:00.597635    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:32:15.195521 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:15.195481    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:32:18.026333 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.026299    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"]
Jul 28 20:32:18.026735 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.026724    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e643e99-97b2-43bb-bba8-917839137789" containerName="authorino"
Jul 28 20:32:18.026777 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.026737    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e643e99-97b2-43bb-bba8-917839137789" containerName="authorino"
Jul 28 20:32:18.026831 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.026821    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e643e99-97b2-43bb-bba8-917839137789" containerName="authorino"
Jul 28 20:32:18.029480 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.029463    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.033920 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.033896    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-35ec9396-kserve-self-signed-certs\""
Jul 28 20:32:18.053449 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.053410    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"]
Jul 28 20:32:18.158482 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.158443    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-home\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.158482 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.158481    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-dshm\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.158786 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.158582    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-tmp-dir\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.158786 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.158649    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-model-cache\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.158786 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.158688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/065d6a77-f10b-4f2b-9553-81a8d8f0b180-tls-certs\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.158932 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.158832    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnvpm\" (UniqueName: \"kubernetes.io/projected/065d6a77-f10b-4f2b-9553-81a8d8f0b180-kube-api-access-hnvpm\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.259629 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.259536    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnvpm\" (UniqueName: \"kubernetes.io/projected/065d6a77-f10b-4f2b-9553-81a8d8f0b180-kube-api-access-hnvpm\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.259816 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.259659    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-home\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.259816 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.259685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-dshm\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.259816 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.259723    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-tmp-dir\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.259816 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.259767    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-model-cache\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.259816 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.259801    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/065d6a77-f10b-4f2b-9553-81a8d8f0b180-tls-certs\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.260124 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.260105    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-tmp-dir\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.260171 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.260156    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-home\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.264766 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.260438    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-model-cache\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.264766 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.262430    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-dshm\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.264766 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.262867    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/065d6a77-f10b-4f2b-9553-81a8d8f0b180-tls-certs\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.276190 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.276151    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnvpm\" (UniqueName: \"kubernetes.io/projected/065d6a77-f10b-4f2b-9553-81a8d8f0b180-kube-api-access-hnvpm\") pod \"e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") " pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.339150 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.339052    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:18.484467 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.484433    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"]
Jul 28 20:32:18.488165 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:32:18.488136    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod065d6a77_f10b_4f2b_9553_81a8d8f0b180.slice/crio-6df0029b75eda4d65d1ce77c55ad7e3acaa29a6e0eab6771fbaf01fc789a480f WatchSource:0}: Error finding container 6df0029b75eda4d65d1ce77c55ad7e3acaa29a6e0eab6771fbaf01fc789a480f: Status 404 returned error can't find the container with id 6df0029b75eda4d65d1ce77c55ad7e3acaa29a6e0eab6771fbaf01fc789a480f
Jul 28 20:32:18.494825 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:18.494797    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9" event={"ID":"065d6a77-f10b-4f2b-9553-81a8d8f0b180","Type":"ContainerStarted","Data":"6df0029b75eda4d65d1ce77c55ad7e3acaa29a6e0eab6771fbaf01fc789a480f"}
Jul 28 20:32:19.502025 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:19.501979    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9" event={"ID":"065d6a77-f10b-4f2b-9553-81a8d8f0b180","Type":"ContainerStarted","Data":"817ef09ee30024c4acacfe5cf1a5f7dbd36f3f68f741566bef73442c2271133e"}
Jul 28 20:32:19.522458 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:19.522403    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9" podStartSLOduration=2.52238633 podStartE2EDuration="2.52238633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:32:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:32:19.520826992 +0000 UTC m=+1416.762920214" watchObservedRunningTime="2026-07-28 20:32:19.52238633 +0000 UTC m=+1416.764479601"
Jul 28 20:32:28.339778 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:28.339727    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:28.340288 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:28.339802    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:28.352346 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:28.352304    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:28.560694 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:28.560655    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:31.013440 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:31.013401    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:32:40.643669 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:40.643587    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"]
Jul 28 20:32:44.400089 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:44.400041    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:32:55.663103 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:55.663035    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9" podUID="065d6a77-f10b-4f2b-9553-81a8d8f0b180" containerName="main" containerID="cri-o://817ef09ee30024c4acacfe5cf1a5f7dbd36f3f68f741566bef73442c2271133e" gracePeriod=45
Jul 28 20:32:55.699728 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:55.699699    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:32:55.922826 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:55.922760    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:56.040157 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.040117    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-home\") pod \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") "
Jul 28 20:32:56.040354 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.040181    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-tmp-dir\") pod \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") "
Jul 28 20:32:56.040354 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.040246    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/065d6a77-f10b-4f2b-9553-81a8d8f0b180-tls-certs\") pod \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") "
Jul 28 20:32:56.040354 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.040303    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-model-cache\") pod \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") "
Jul 28 20:32:56.040518 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.040382    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnvpm\" (UniqueName: \"kubernetes.io/projected/065d6a77-f10b-4f2b-9553-81a8d8f0b180-kube-api-access-hnvpm\") pod \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") "
Jul 28 20:32:56.040518 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.040417    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-dshm\") pod \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\" (UID: \"065d6a77-f10b-4f2b-9553-81a8d8f0b180\") "
Jul 28 20:32:56.040518 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.040447    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-home" (OuterVolumeSpecName: "home") pod "065d6a77-f10b-4f2b-9553-81a8d8f0b180" (UID: "065d6a77-f10b-4f2b-9553-81a8d8f0b180"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:32:56.040518 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.040469    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "065d6a77-f10b-4f2b-9553-81a8d8f0b180" (UID: "065d6a77-f10b-4f2b-9553-81a8d8f0b180"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:32:56.040714 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.040599    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-model-cache" (OuterVolumeSpecName: "model-cache") pod "065d6a77-f10b-4f2b-9553-81a8d8f0b180" (UID: "065d6a77-f10b-4f2b-9553-81a8d8f0b180"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:32:56.040751 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.040726    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-home\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:32:56.040751 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.040741    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-tmp-dir\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:32:56.040813 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.040749    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-model-cache\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:32:56.042653 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.042630    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-dshm" (OuterVolumeSpecName: "dshm") pod "065d6a77-f10b-4f2b-9553-81a8d8f0b180" (UID: "065d6a77-f10b-4f2b-9553-81a8d8f0b180"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:32:56.042752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.042661    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/065d6a77-f10b-4f2b-9553-81a8d8f0b180-kube-api-access-hnvpm" (OuterVolumeSpecName: "kube-api-access-hnvpm") pod "065d6a77-f10b-4f2b-9553-81a8d8f0b180" (UID: "065d6a77-f10b-4f2b-9553-81a8d8f0b180"). InnerVolumeSpecName "kube-api-access-hnvpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:32:56.042752 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.042673    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/065d6a77-f10b-4f2b-9553-81a8d8f0b180-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "065d6a77-f10b-4f2b-9553-81a8d8f0b180" (UID: "065d6a77-f10b-4f2b-9553-81a8d8f0b180"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:32:56.141680 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.141641    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnvpm\" (UniqueName: \"kubernetes.io/projected/065d6a77-f10b-4f2b-9553-81a8d8f0b180-kube-api-access-hnvpm\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:32:56.141680 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.141674    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/065d6a77-f10b-4f2b-9553-81a8d8f0b180-dshm\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:32:56.141680 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.141683    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/065d6a77-f10b-4f2b-9553-81a8d8f0b180-tls-certs\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:32:56.700725 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.700686    2558 generic.go:356] "Generic (PLEG): container finished" podID="065d6a77-f10b-4f2b-9553-81a8d8f0b180" containerID="817ef09ee30024c4acacfe5cf1a5f7dbd36f3f68f741566bef73442c2271133e" exitCode=0
Jul 28 20:32:56.701167 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.700741    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"
Jul 28 20:32:56.701167 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.700761    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9" event={"ID":"065d6a77-f10b-4f2b-9553-81a8d8f0b180","Type":"ContainerDied","Data":"817ef09ee30024c4acacfe5cf1a5f7dbd36f3f68f741566bef73442c2271133e"}
Jul 28 20:32:56.701167 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.700810    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9" event={"ID":"065d6a77-f10b-4f2b-9553-81a8d8f0b180","Type":"ContainerDied","Data":"6df0029b75eda4d65d1ce77c55ad7e3acaa29a6e0eab6771fbaf01fc789a480f"}
Jul 28 20:32:56.701167 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.700828    2558 scope.go:117] "RemoveContainer" containerID="817ef09ee30024c4acacfe5cf1a5f7dbd36f3f68f741566bef73442c2271133e"
Jul 28 20:32:56.710480 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.710460    2558 scope.go:117] "RemoveContainer" containerID="817ef09ee30024c4acacfe5cf1a5f7dbd36f3f68f741566bef73442c2271133e"
Jul 28 20:32:56.710806 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:32:56.710781    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"817ef09ee30024c4acacfe5cf1a5f7dbd36f3f68f741566bef73442c2271133e\": container with ID starting with 817ef09ee30024c4acacfe5cf1a5f7dbd36f3f68f741566bef73442c2271133e not found: ID does not exist" containerID="817ef09ee30024c4acacfe5cf1a5f7dbd36f3f68f741566bef73442c2271133e"
Jul 28 20:32:56.710898 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.710821    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"817ef09ee30024c4acacfe5cf1a5f7dbd36f3f68f741566bef73442c2271133e"} err="failed to get container status \"817ef09ee30024c4acacfe5cf1a5f7dbd36f3f68f741566bef73442c2271133e\": rpc error: code = NotFound desc = could not find container \"817ef09ee30024c4acacfe5cf1a5f7dbd36f3f68f741566bef73442c2271133e\": container with ID starting with 817ef09ee30024c4acacfe5cf1a5f7dbd36f3f68f741566bef73442c2271133e not found: ID does not exist"
Jul 28 20:32:56.723944 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.723916    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"]
Jul 28 20:32:56.727733 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:56.727707    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-35ec9396-kserve-749757d987-nlmw9"]
Jul 28 20:32:57.340526 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:32:57.340491    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="065d6a77-f10b-4f2b-9553-81a8d8f0b180" path="/var/lib/kubelet/pods/065d6a77-f10b-4f2b-9553-81a8d8f0b180/volumes"
Jul 28 20:33:04.703513 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:33:04.703468    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:33:16.988232 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:33:16.988190    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:33:26.402273 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:33:26.402233    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:33:35.388342 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:33:35.388304    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:33:43.286423 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:33:43.286391    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:33:43.286911 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:33:43.286393    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:33:43.289889 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:33:43.289864    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:33:53.913568 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:33:53.913520    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:34:03.395260 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:34:03.395214    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:34:15.902113 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:34:15.902033    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:34:24.892094 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:34:24.892050    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:34:34.433632 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:34:34.433585    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:34:49.489020 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:34:49.488983    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:35:01.094092 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:35:01.094052    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:35:18.402895 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:35:18.402842    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:35:29.996178 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:35:29.996138    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:35:40.293022 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:35:40.292942    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:35:47.989477 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:35:47.989440    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:36:01.187804 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:36:01.187760    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:36:13.894489 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:36:13.894451    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:36:30.496070 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:36:30.496028    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:36:57.695003 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:36:57.694965    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:37:07.792903 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:37:07.792825    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:37:32.710999 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:37:32.710955    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:37:43.086479 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:37:43.086437    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:38:07.199303 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:38:07.199262    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:38:17.992783 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:38:17.992743    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:38:42.095260 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:38:42.095221    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:38:43.321973 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:38:43.321939    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:38:43.327191 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:38:43.327164    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:40:05.299072 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:40:05.298986    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:40:17.577263 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:40:17.577225    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:40:27.886965 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:40:27.886924    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:40:38.095493 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:40:38.095457    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:41:20.005690 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:41:20.005644    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:41:52.492265 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:41:52.492220    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:42:26.789866 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:42:26.789825    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:42:42.495595 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:42:42.495535    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:43:00.999292 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:43:00.999241    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:43:09.683162 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:43:09.683079    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:43:43.357078 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:43:43.357047    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:43:43.362493 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:43:43.362464    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:43:58.837909 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:43:58.837866    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:44:09.002347 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:44:09.002307    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:45:01.991720 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:45:01.991676    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:45:32.803695 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:45:32.803654    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:46:05.400174 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.400092    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"]
Jul 28 20:46:05.400642 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.400509    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="065d6a77-f10b-4f2b-9553-81a8d8f0b180" containerName="main"
Jul 28 20:46:05.400642 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.400521    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="065d6a77-f10b-4f2b-9553-81a8d8f0b180" containerName="main"
Jul 28 20:46:05.400642 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.400604    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="065d6a77-f10b-4f2b-9553-81a8d8f0b180" containerName="main"
Jul 28 20:46:05.403782 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.403761    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.408573 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.408523    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d82230\"/\"test-model-d82230-kserve-self-signed-certs\""
Jul 28 20:46:05.408573 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.408545    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d82230\"/\"openshift-service-ca.crt\""
Jul 28 20:46:05.408807 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.408571    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d82230\"/\"default-dockercfg-6hscp\""
Jul 28 20:46:05.408807 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.408734    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d82230\"/\"kube-root-ca.crt\""
Jul 28 20:46:05.415709 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.415686    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"]
Jul 28 20:46:05.501850 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.501805    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c331a19a-5715-48dc-8226-fe737b1c349d-tls-certs\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.502030 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.501872    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tktsv\" (UniqueName: \"kubernetes.io/projected/c331a19a-5715-48dc-8226-fe737b1c349d-kube-api-access-tktsv\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.502030 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.501898    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-model-cache\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.502030 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.501926    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-tmp-dir\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.502030 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.501947    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-home\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.502030 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.502010    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-dshm\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.603397 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.603347    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-tmp-dir\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.603397 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.603392    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-home\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.603687 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.603413    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-dshm\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.603687 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.603479    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c331a19a-5715-48dc-8226-fe737b1c349d-tls-certs\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.603687 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.603513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tktsv\" (UniqueName: \"kubernetes.io/projected/c331a19a-5715-48dc-8226-fe737b1c349d-kube-api-access-tktsv\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.603687 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.603533    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-model-cache\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.603898 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.603861    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-tmp-dir\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.603954 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.603927    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-home\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.603997 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.603966    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-model-cache\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.605819 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.605796    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-dshm\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.606025 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.606005    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c331a19a-5715-48dc-8226-fe737b1c349d-tls-certs\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.612388 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.612361    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tktsv\" (UniqueName: \"kubernetes.io/projected/c331a19a-5715-48dc-8226-fe737b1c349d-kube-api-access-tktsv\") pod \"test-model-d82230-kserve-599bf4dfc9-fdwnq\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") " pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.714603 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.714473    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:05.895184 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:46:05.895150    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc331a19a_5715_48dc_8226_fe737b1c349d.slice/crio-0696d01a74273c54676bd6a1f7f10ced8a15573a15fc9c7af6ecbcd766346c53 WatchSource:0}: Error finding container 0696d01a74273c54676bd6a1f7f10ced8a15573a15fc9c7af6ecbcd766346c53: Status 404 returned error can't find the container with id 0696d01a74273c54676bd6a1f7f10ced8a15573a15fc9c7af6ecbcd766346c53
Jul 28 20:46:05.895333 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:05.895209    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"]
Jul 28 20:46:06.728155 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:06.728114    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq" event={"ID":"c331a19a-5715-48dc-8226-fe737b1c349d","Type":"ContainerStarted","Data":"806ef352a8697ca8a57f3e8b53b71e9c8b8f7158cfd6e2413e1d9ff7a0217a1f"}
Jul 28 20:46:06.728602 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:06.728166    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq" event={"ID":"c331a19a-5715-48dc-8226-fe737b1c349d","Type":"ContainerStarted","Data":"0696d01a74273c54676bd6a1f7f10ced8a15573a15fc9c7af6ecbcd766346c53"}
Jul 28 20:46:06.748337 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:06.748281    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq" podStartSLOduration=1.748262864 podStartE2EDuration="1.748262864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:06.747053319 +0000 UTC m=+2243.989146543" watchObservedRunningTime="2026-07-28 20:46:06.748262864 +0000 UTC m=+2243.990356065"
Jul 28 20:46:09.798948 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:09.798907    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:46:15.715492 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:15.715455    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:15.715492 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:15.715504    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:15.727936 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:15.727906    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:15.784817 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:15.784789    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:46:27.003057 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.003020    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"]
Jul 28 20:46:27.006331 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.006312    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.010267 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.010235    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a7eeff\"/\"kube-root-ca.crt\""
Jul 28 20:46:27.011516 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.011488    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a7eeff\"/\"test-model-a7eeff-kserve-self-signed-certs\""
Jul 28 20:46:27.011675 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.011543    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a7eeff\"/\"openshift-service-ca.crt\""
Jul 28 20:46:27.011675 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.011595    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a7eeff\"/\"default-dockercfg-nqfrz\""
Jul 28 20:46:27.017541 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.017514    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"]
Jul 28 20:46:27.125236 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.125198    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-home\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.125422 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.125280    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk2x8\" (UniqueName: \"kubernetes.io/projected/7e1ff993-db2a-40a4-87a0-9352d9db9c86-kube-api-access-bk2x8\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.125422 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.125315    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-dshm\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.125422 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.125339    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-model-cache\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.125529 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.125440    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e1ff993-db2a-40a4-87a0-9352d9db9c86-tls-certs\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.125529 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.125487    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-tmp-dir\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.226636 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.226597    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e1ff993-db2a-40a4-87a0-9352d9db9c86-tls-certs\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.226636 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.226644    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-tmp-dir\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.226875 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.226672    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-home\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.226875 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.226755    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk2x8\" (UniqueName: \"kubernetes.io/projected/7e1ff993-db2a-40a4-87a0-9352d9db9c86-kube-api-access-bk2x8\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.226875 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.226794    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-dshm\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.226875 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.226823    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-model-cache\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.227179 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.227153    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-tmp-dir\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.227255 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.227205    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-home\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.227255 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.227221    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-model-cache\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.229161 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.229137    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-dshm\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.229271 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.229161    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e1ff993-db2a-40a4-87a0-9352d9db9c86-tls-certs\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.236896 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.236864    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk2x8\" (UniqueName: \"kubernetes.io/projected/7e1ff993-db2a-40a4-87a0-9352d9db9c86-kube-api-access-bk2x8\") pod \"test-model-a7eeff-kserve-76bd996465-h6kwd\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") " pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.317194 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.317104    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:27.463523 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.463495    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"]
Jul 28 20:46:27.466568 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:46:27.466525    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e1ff993_db2a_40a4_87a0_9352d9db9c86.slice/crio-d38e08cd1749b5a576859a3a6f5247c1b51c175417489ccdf51bdd302579cdc5 WatchSource:0}: Error finding container d38e08cd1749b5a576859a3a6f5247c1b51c175417489ccdf51bdd302579cdc5: Status 404 returned error can't find the container with id d38e08cd1749b5a576859a3a6f5247c1b51c175417489ccdf51bdd302579cdc5
Jul 28 20:46:27.834103 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.834059    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd" event={"ID":"7e1ff993-db2a-40a4-87a0-9352d9db9c86","Type":"ContainerStarted","Data":"3ff9e463f3c61967c7147cc9dd526b67c968b66d2e72f74da10d9e436e844640"}
Jul 28 20:46:27.834103 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.834112    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd" event={"ID":"7e1ff993-db2a-40a4-87a0-9352d9db9c86","Type":"ContainerStarted","Data":"d38e08cd1749b5a576859a3a6f5247c1b51c175417489ccdf51bdd302579cdc5"}
Jul 28 20:46:27.852917 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:27.852860    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd" podStartSLOduration=1.852840462 podStartE2EDuration="1.852840462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:46:27.851423095 +0000 UTC m=+2265.093516318" watchObservedRunningTime="2026-07-28 20:46:27.852840462 +0000 UTC m=+2265.094933751"
Jul 28 20:46:33.096189 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:33.096138    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:46:37.318275 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:37.318221    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:37.318275 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:37.318286    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:37.331262 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:37.331222    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:46:37.897020 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:46:37.896984    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:47:15.479579 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:15.479506    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"]
Jul 28 20:47:16.388180 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:16.388143    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"]
Jul 28 20:47:20.402835 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:20.402790    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:47:30.499213 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.499141    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq" podUID="c331a19a-5715-48dc-8226-fe737b1c349d" containerName="main" containerID="cri-o://806ef352a8697ca8a57f3e8b53b71e9c8b8f7158cfd6e2413e1d9ff7a0217a1f" gracePeriod=45
Jul 28 20:47:30.771707 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.771671    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:47:30.875093 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.875051    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-model-cache\") pod \"c331a19a-5715-48dc-8226-fe737b1c349d\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") "
Jul 28 20:47:30.875296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.875105    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-home\") pod \"c331a19a-5715-48dc-8226-fe737b1c349d\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") "
Jul 28 20:47:30.875296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.875128    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-dshm\") pod \"c331a19a-5715-48dc-8226-fe737b1c349d\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") "
Jul 28 20:47:30.875296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.875159    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-tmp-dir\") pod \"c331a19a-5715-48dc-8226-fe737b1c349d\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") "
Jul 28 20:47:30.875296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.875230    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c331a19a-5715-48dc-8226-fe737b1c349d-tls-certs\") pod \"c331a19a-5715-48dc-8226-fe737b1c349d\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") "
Jul 28 20:47:30.875296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.875272    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tktsv\" (UniqueName: \"kubernetes.io/projected/c331a19a-5715-48dc-8226-fe737b1c349d-kube-api-access-tktsv\") pod \"c331a19a-5715-48dc-8226-fe737b1c349d\" (UID: \"c331a19a-5715-48dc-8226-fe737b1c349d\") "
Jul 28 20:47:30.875588 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.875369    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-home" (OuterVolumeSpecName: "home") pod "c331a19a-5715-48dc-8226-fe737b1c349d" (UID: "c331a19a-5715-48dc-8226-fe737b1c349d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:47:30.875588 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.875391    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-model-cache" (OuterVolumeSpecName: "model-cache") pod "c331a19a-5715-48dc-8226-fe737b1c349d" (UID: "c331a19a-5715-48dc-8226-fe737b1c349d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:47:30.875588 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.875549    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c331a19a-5715-48dc-8226-fe737b1c349d" (UID: "c331a19a-5715-48dc-8226-fe737b1c349d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:47:30.875708 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.875658    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-model-cache\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:47:30.875708 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.875674    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-home\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:47:30.875708 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.875683    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-tmp-dir\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:47:30.877475 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.877454    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-dshm" (OuterVolumeSpecName: "dshm") pod "c331a19a-5715-48dc-8226-fe737b1c349d" (UID: "c331a19a-5715-48dc-8226-fe737b1c349d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:47:30.877588 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.877499    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c331a19a-5715-48dc-8226-fe737b1c349d-kube-api-access-tktsv" (OuterVolumeSpecName: "kube-api-access-tktsv") pod "c331a19a-5715-48dc-8226-fe737b1c349d" (UID: "c331a19a-5715-48dc-8226-fe737b1c349d"). InnerVolumeSpecName "kube-api-access-tktsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:47:30.877588 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.877510    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c331a19a-5715-48dc-8226-fe737b1c349d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c331a19a-5715-48dc-8226-fe737b1c349d" (UID: "c331a19a-5715-48dc-8226-fe737b1c349d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:47:30.977236 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.977193    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tktsv\" (UniqueName: \"kubernetes.io/projected/c331a19a-5715-48dc-8226-fe737b1c349d-kube-api-access-tktsv\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:47:30.977236 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.977229    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c331a19a-5715-48dc-8226-fe737b1c349d-dshm\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:47:30.977236 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:30.977241    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c331a19a-5715-48dc-8226-fe737b1c349d-tls-certs\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:47:31.174711 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.174620    2558 generic.go:356] "Generic (PLEG): container finished" podID="c331a19a-5715-48dc-8226-fe737b1c349d" containerID="806ef352a8697ca8a57f3e8b53b71e9c8b8f7158cfd6e2413e1d9ff7a0217a1f" exitCode=0
Jul 28 20:47:31.174711 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.174661    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"
Jul 28 20:47:31.174711 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.174683    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq" event={"ID":"c331a19a-5715-48dc-8226-fe737b1c349d","Type":"ContainerDied","Data":"806ef352a8697ca8a57f3e8b53b71e9c8b8f7158cfd6e2413e1d9ff7a0217a1f"}
Jul 28 20:47:31.174955 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.174732    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq" event={"ID":"c331a19a-5715-48dc-8226-fe737b1c349d","Type":"ContainerDied","Data":"0696d01a74273c54676bd6a1f7f10ced8a15573a15fc9c7af6ecbcd766346c53"}
Jul 28 20:47:31.174955 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.174756    2558 scope.go:117] "RemoveContainer" containerID="806ef352a8697ca8a57f3e8b53b71e9c8b8f7158cfd6e2413e1d9ff7a0217a1f"
Jul 28 20:47:31.184399 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.184377    2558 scope.go:117] "RemoveContainer" containerID="806ef352a8697ca8a57f3e8b53b71e9c8b8f7158cfd6e2413e1d9ff7a0217a1f"
Jul 28 20:47:31.184707 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:47:31.184688    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"806ef352a8697ca8a57f3e8b53b71e9c8b8f7158cfd6e2413e1d9ff7a0217a1f\": container with ID starting with 806ef352a8697ca8a57f3e8b53b71e9c8b8f7158cfd6e2413e1d9ff7a0217a1f not found: ID does not exist" containerID="806ef352a8697ca8a57f3e8b53b71e9c8b8f7158cfd6e2413e1d9ff7a0217a1f"
Jul 28 20:47:31.184774 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.184716    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"806ef352a8697ca8a57f3e8b53b71e9c8b8f7158cfd6e2413e1d9ff7a0217a1f"} err="failed to get container status \"806ef352a8697ca8a57f3e8b53b71e9c8b8f7158cfd6e2413e1d9ff7a0217a1f\": rpc error: code = NotFound desc = could not find container \"806ef352a8697ca8a57f3e8b53b71e9c8b8f7158cfd6e2413e1d9ff7a0217a1f\": container with ID starting with 806ef352a8697ca8a57f3e8b53b71e9c8b8f7158cfd6e2413e1d9ff7a0217a1f not found: ID does not exist"
Jul 28 20:47:31.202397 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.202362    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"]
Jul 28 20:47:31.206868 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.206833    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-d82230/test-model-d82230-kserve-599bf4dfc9-fdwnq"]
Jul 28 20:47:31.341011 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.340976    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c331a19a-5715-48dc-8226-fe737b1c349d" path="/var/lib/kubelet/pods/c331a19a-5715-48dc-8226-fe737b1c349d/volumes"
Jul 28 20:47:31.408242 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.408205    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd" podUID="7e1ff993-db2a-40a4-87a0-9352d9db9c86" containerName="main" containerID="cri-o://3ff9e463f3c61967c7147cc9dd526b67c968b66d2e72f74da10d9e436e844640" gracePeriod=45
Jul 28 20:47:31.659176 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.659139    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:47:31.787317 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.787225    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-tmp-dir\") pod \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") "
Jul 28 20:47:31.787317 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.787289    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-dshm\") pod \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") "
Jul 28 20:47:31.787317 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.787318    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e1ff993-db2a-40a4-87a0-9352d9db9c86-tls-certs\") pod \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") "
Jul 28 20:47:31.787653 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.787335    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-home\") pod \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") "
Jul 28 20:47:31.787653 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.787374    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bk2x8\" (UniqueName: \"kubernetes.io/projected/7e1ff993-db2a-40a4-87a0-9352d9db9c86-kube-api-access-bk2x8\") pod \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") "
Jul 28 20:47:31.787653 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.787392    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-model-cache\") pod \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\" (UID: \"7e1ff993-db2a-40a4-87a0-9352d9db9c86\") "
Jul 28 20:47:31.787653 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.787552    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e1ff993-db2a-40a4-87a0-9352d9db9c86" (UID: "7e1ff993-db2a-40a4-87a0-9352d9db9c86"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:47:31.787850 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.787718    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-home" (OuterVolumeSpecName: "home") pod "7e1ff993-db2a-40a4-87a0-9352d9db9c86" (UID: "7e1ff993-db2a-40a4-87a0-9352d9db9c86"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:47:31.787850 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.787812    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-tmp-dir\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:47:31.787850 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.787835    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-home\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:47:31.787978 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.787918    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e1ff993-db2a-40a4-87a0-9352d9db9c86" (UID: "7e1ff993-db2a-40a4-87a0-9352d9db9c86"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:47:31.789725 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.789692    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-dshm" (OuterVolumeSpecName: "dshm") pod "7e1ff993-db2a-40a4-87a0-9352d9db9c86" (UID: "7e1ff993-db2a-40a4-87a0-9352d9db9c86"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:47:31.789725 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.789708    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e1ff993-db2a-40a4-87a0-9352d9db9c86-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e1ff993-db2a-40a4-87a0-9352d9db9c86" (UID: "7e1ff993-db2a-40a4-87a0-9352d9db9c86"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:47:31.789877 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.789802    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e1ff993-db2a-40a4-87a0-9352d9db9c86-kube-api-access-bk2x8" (OuterVolumeSpecName: "kube-api-access-bk2x8") pod "7e1ff993-db2a-40a4-87a0-9352d9db9c86" (UID: "7e1ff993-db2a-40a4-87a0-9352d9db9c86"). InnerVolumeSpecName "kube-api-access-bk2x8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:47:31.889096 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.889053    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-dshm\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:47:31.889096 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.889091    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e1ff993-db2a-40a4-87a0-9352d9db9c86-tls-certs\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:47:31.889308 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.889109    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bk2x8\" (UniqueName: \"kubernetes.io/projected/7e1ff993-db2a-40a4-87a0-9352d9db9c86-kube-api-access-bk2x8\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:47:31.889308 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:31.889125    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e1ff993-db2a-40a4-87a0-9352d9db9c86-model-cache\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:47:32.182573 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:32.182462    2558 generic.go:356] "Generic (PLEG): container finished" podID="7e1ff993-db2a-40a4-87a0-9352d9db9c86" containerID="3ff9e463f3c61967c7147cc9dd526b67c968b66d2e72f74da10d9e436e844640" exitCode=0
Jul 28 20:47:32.182573 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:32.182502    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd" event={"ID":"7e1ff993-db2a-40a4-87a0-9352d9db9c86","Type":"ContainerDied","Data":"3ff9e463f3c61967c7147cc9dd526b67c968b66d2e72f74da10d9e436e844640"}
Jul 28 20:47:32.182573 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:32.182509    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"
Jul 28 20:47:32.182573 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:32.182531    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd" event={"ID":"7e1ff993-db2a-40a4-87a0-9352d9db9c86","Type":"ContainerDied","Data":"d38e08cd1749b5a576859a3a6f5247c1b51c175417489ccdf51bdd302579cdc5"}
Jul 28 20:47:32.182573 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:32.182547    2558 scope.go:117] "RemoveContainer" containerID="3ff9e463f3c61967c7147cc9dd526b67c968b66d2e72f74da10d9e436e844640"
Jul 28 20:47:32.191847 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:32.191825    2558 scope.go:117] "RemoveContainer" containerID="3ff9e463f3c61967c7147cc9dd526b67c968b66d2e72f74da10d9e436e844640"
Jul 28 20:47:32.192137 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:47:32.192118    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ff9e463f3c61967c7147cc9dd526b67c968b66d2e72f74da10d9e436e844640\": container with ID starting with 3ff9e463f3c61967c7147cc9dd526b67c968b66d2e72f74da10d9e436e844640 not found: ID does not exist" containerID="3ff9e463f3c61967c7147cc9dd526b67c968b66d2e72f74da10d9e436e844640"
Jul 28 20:47:32.192182 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:32.192147    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ff9e463f3c61967c7147cc9dd526b67c968b66d2e72f74da10d9e436e844640"} err="failed to get container status \"3ff9e463f3c61967c7147cc9dd526b67c968b66d2e72f74da10d9e436e844640\": rpc error: code = NotFound desc = could not find container \"3ff9e463f3c61967c7147cc9dd526b67c968b66d2e72f74da10d9e436e844640\": container with ID starting with 3ff9e463f3c61967c7147cc9dd526b67c968b66d2e72f74da10d9e436e844640 not found: ID does not exist"
Jul 28 20:47:32.207187 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:32.207156    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"]
Jul 28 20:47:32.210002 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:32.209971    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a7eeff/test-model-a7eeff-kserve-76bd996465-h6kwd"]
Jul 28 20:47:33.340578 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:33.340535    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e1ff993-db2a-40a4-87a0-9352d9db9c86" path="/var/lib/kubelet/pods/7e1ff993-db2a-40a4-87a0-9352d9db9c86/volumes"
Jul 28 20:47:35.904615 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:35.904532    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:47:48.801095 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:47:48.801055    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:48:01.794584 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:01.794527    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:48:35.751168 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:35.750350    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:48:43.390183 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:43.390156    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:48:43.400193 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:43.400165    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:48:44.764258 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.764218    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds"]
Jul 28 20:48:44.764701 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.764685    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c331a19a-5715-48dc-8226-fe737b1c349d" containerName="main"
Jul 28 20:48:44.764746 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.764703    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c331a19a-5715-48dc-8226-fe737b1c349d" containerName="main"
Jul 28 20:48:44.764746 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.764731    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e1ff993-db2a-40a4-87a0-9352d9db9c86" containerName="main"
Jul 28 20:48:44.764746 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.764739    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e1ff993-db2a-40a4-87a0-9352d9db9c86" containerName="main"
Jul 28 20:48:44.764842 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.764800    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c331a19a-5715-48dc-8226-fe737b1c349d" containerName="main"
Jul 28 20:48:44.764842 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.764814    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e1ff993-db2a-40a4-87a0-9352d9db9c86" containerName="main"
Jul 28 20:48:44.768238 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.768204    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds"
Jul 28 20:48:44.773747 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.773717    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gcrkz\""
Jul 28 20:48:44.774086 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.774059    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:48:44.788128 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.788093    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds"]
Jul 28 20:48:44.791293 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.791261    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d802801b-3ac0-4a48-b9bb-486e7a3ca2db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds\" (UID: \"d802801b-3ac0-4a48-b9bb-486e7a3ca2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds"
Jul 28 20:48:44.892310 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.892265    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d802801b-3ac0-4a48-b9bb-486e7a3ca2db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds\" (UID: \"d802801b-3ac0-4a48-b9bb-486e7a3ca2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds"
Jul 28 20:48:44.892939 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:44.892917    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d802801b-3ac0-4a48-b9bb-486e7a3ca2db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds\" (UID: \"d802801b-3ac0-4a48-b9bb-486e7a3ca2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds"
Jul 28 20:48:45.079291 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:45.079202    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds"
Jul 28 20:48:45.239696 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:45.239661    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds"]
Jul 28 20:48:45.241179 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:48:45.241145    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd802801b_3ac0_4a48_b9bb_486e7a3ca2db.slice/crio-8eddce02adfb344bf5a2e4ecde5b9f6ca3bf8edbb50e75249fc7462a9a7dbefd WatchSource:0}: Error finding container 8eddce02adfb344bf5a2e4ecde5b9f6ca3bf8edbb50e75249fc7462a9a7dbefd: Status 404 returned error can't find the container with id 8eddce02adfb344bf5a2e4ecde5b9f6ca3bf8edbb50e75249fc7462a9a7dbefd
Jul 28 20:48:45.243167 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:45.243149    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:48:45.561402 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:45.561369    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds" event={"ID":"d802801b-3ac0-4a48-b9bb-486e7a3ca2db","Type":"ContainerStarted","Data":"8eddce02adfb344bf5a2e4ecde5b9f6ca3bf8edbb50e75249fc7462a9a7dbefd"}
Jul 28 20:48:52.600881 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:52.600840    2558 generic.go:356] "Generic (PLEG): container finished" podID="d802801b-3ac0-4a48-b9bb-486e7a3ca2db" containerID="4077ab3e3572653059dfbab9b98b24a9518192980cf03239efec38ab1094bd36" exitCode=0
Jul 28 20:48:52.601296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:52.600936    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds" event={"ID":"d802801b-3ac0-4a48-b9bb-486e7a3ca2db","Type":"ContainerDied","Data":"4077ab3e3572653059dfbab9b98b24a9518192980cf03239efec38ab1094bd36"}
Jul 28 20:48:53.742838 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:53.742800    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds"
Jul 28 20:48:53.883175 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:53.883074    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d802801b-3ac0-4a48-b9bb-486e7a3ca2db-maas-api-service-ca\") pod \"d802801b-3ac0-4a48-b9bb-486e7a3ca2db\" (UID: \"d802801b-3ac0-4a48-b9bb-486e7a3ca2db\") "
Jul 28 20:48:53.883456 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:53.883433    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d802801b-3ac0-4a48-b9bb-486e7a3ca2db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d802801b-3ac0-4a48-b9bb-486e7a3ca2db" (UID: "d802801b-3ac0-4a48-b9bb-486e7a3ca2db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:48:53.984092 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:53.984052    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d802801b-3ac0-4a48-b9bb-486e7a3ca2db-maas-api-service-ca\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:48:54.612762 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:54.612728    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds"
Jul 28 20:48:54.612762 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:54.612745    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds" event={"ID":"d802801b-3ac0-4a48-b9bb-486e7a3ca2db","Type":"ContainerDied","Data":"8eddce02adfb344bf5a2e4ecde5b9f6ca3bf8edbb50e75249fc7462a9a7dbefd"}
Jul 28 20:48:54.612978 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:48:54.612791    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8eddce02adfb344bf5a2e4ecde5b9f6ca3bf8edbb50e75249fc7462a9a7dbefd"
Jul 28 20:49:05.512711 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:05.512623    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds"]
Jul 28 20:49:05.516816 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:05.516783    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-65eee9-7p7ds"]
Jul 28 20:49:07.340701 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:07.340659    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d802801b-3ac0-4a48-b9bb-486e7a3ca2db" path="/var/lib/kubelet/pods/d802801b-3ac0-4a48-b9bb-486e7a3ca2db/volumes"
Jul 28 20:49:10.703290 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:10.703238    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:49:13.301234 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:13.301187    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7"]
Jul 28 20:49:13.301809 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:13.301787    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d802801b-3ac0-4a48-b9bb-486e7a3ca2db" containerName="revoke-keys"
Jul 28 20:49:13.301895 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:13.301812    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d802801b-3ac0-4a48-b9bb-486e7a3ca2db" containerName="revoke-keys"
Jul 28 20:49:13.301968 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:13.301955    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d802801b-3ac0-4a48-b9bb-486e7a3ca2db" containerName="revoke-keys"
Jul 28 20:49:13.305503 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:13.305481    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7"
Jul 28 20:49:13.308706 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:13.308683    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:49:13.308841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:13.308714    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gcrkz\""
Jul 28 20:49:13.315599 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:13.315569    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7"]
Jul 28 20:49:13.378856 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:13.378813    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cd17f3e-ca67-4005-802f-57d6224f4dd2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-42ff09-44kq7\" (UID: \"9cd17f3e-ca67-4005-802f-57d6224f4dd2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7"
Jul 28 20:49:13.480183 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:13.480147    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cd17f3e-ca67-4005-802f-57d6224f4dd2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-42ff09-44kq7\" (UID: \"9cd17f3e-ca67-4005-802f-57d6224f4dd2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7"
Jul 28 20:49:13.480924 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:13.480899    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cd17f3e-ca67-4005-802f-57d6224f4dd2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-42ff09-44kq7\" (UID: \"9cd17f3e-ca67-4005-802f-57d6224f4dd2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7"
Jul 28 20:49:13.617183 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:13.617093    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7"
Jul 28 20:49:13.799363 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:13.799335    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7"]
Jul 28 20:49:13.801490 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:49:13.801461    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cd17f3e_ca67_4005_802f_57d6224f4dd2.slice/crio-f843da3d4e69fe1b7b5969b60aed39253ab22b86eead764043f4c3c350b61759 WatchSource:0}: Error finding container f843da3d4e69fe1b7b5969b60aed39253ab22b86eead764043f4c3c350b61759: Status 404 returned error can't find the container with id f843da3d4e69fe1b7b5969b60aed39253ab22b86eead764043f4c3c350b61759
Jul 28 20:49:14.724438 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:14.724399    2558 generic.go:356] "Generic (PLEG): container finished" podID="9cd17f3e-ca67-4005-802f-57d6224f4dd2" containerID="cb8b197ec90c7096cb7d90d91cda0f2fcf4b6290003ad00f96ebf547994fada2" exitCode=0
Jul 28 20:49:14.724836 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:14.724485    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7" event={"ID":"9cd17f3e-ca67-4005-802f-57d6224f4dd2","Type":"ContainerDied","Data":"cb8b197ec90c7096cb7d90d91cda0f2fcf4b6290003ad00f96ebf547994fada2"}
Jul 28 20:49:14.724836 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:14.724542    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7" event={"ID":"9cd17f3e-ca67-4005-802f-57d6224f4dd2","Type":"ContainerStarted","Data":"f843da3d4e69fe1b7b5969b60aed39253ab22b86eead764043f4c3c350b61759"}
Jul 28 20:49:15.869530 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:15.869498    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7"
Jul 28 20:49:15.904703 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:15.904666    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cd17f3e-ca67-4005-802f-57d6224f4dd2-maas-api-service-ca\") pod \"9cd17f3e-ca67-4005-802f-57d6224f4dd2\" (UID: \"9cd17f3e-ca67-4005-802f-57d6224f4dd2\") "
Jul 28 20:49:15.905079 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:15.905050    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cd17f3e-ca67-4005-802f-57d6224f4dd2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9cd17f3e-ca67-4005-802f-57d6224f4dd2" (UID: "9cd17f3e-ca67-4005-802f-57d6224f4dd2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:49:16.005841 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:16.005805    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cd17f3e-ca67-4005-802f-57d6224f4dd2-maas-api-service-ca\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:49:16.735393 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:16.735356    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7"
Jul 28 20:49:16.735393 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:16.735380    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7" event={"ID":"9cd17f3e-ca67-4005-802f-57d6224f4dd2","Type":"ContainerDied","Data":"f843da3d4e69fe1b7b5969b60aed39253ab22b86eead764043f4c3c350b61759"}
Jul 28 20:49:16.735650 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:16.735428    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f843da3d4e69fe1b7b5969b60aed39253ab22b86eead764043f4c3c350b61759"
Jul 28 20:49:23.541538 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:23.541504    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7"]
Jul 28 20:49:23.544439 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:23.544405    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-42ff09-44kq7"]
Jul 28 20:49:25.340309 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:25.340275    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cd17f3e-ca67-4005-802f-57d6224f4dd2" path="/var/lib/kubelet/pods/9cd17f3e-ca67-4005-802f-57d6224f4dd2/volumes"
Jul 28 20:49:25.399506 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:25.399468    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:49:39.706661 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:49:39.706620    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:50:11.898099 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:11.898055    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:50:38.994776 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:38.994688    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"]
Jul 28 20:50:38.996886 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:38.995101    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cd17f3e-ca67-4005-802f-57d6224f4dd2" containerName="revoke-keys"
Jul 28 20:50:38.996886 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:38.995113    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cd17f3e-ca67-4005-802f-57d6224f4dd2" containerName="revoke-keys"
Jul 28 20:50:38.996886 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:38.995197    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cd17f3e-ca67-4005-802f-57d6224f4dd2" containerName="revoke-keys"
Jul 28 20:50:38.997935 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:38.997913    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.000734 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.000713    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-cc370c\"/\"auth-test-model-cc370c-kserve-self-signed-certs\""
Jul 28 20:50:39.001646 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.001622    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-cc370c\"/\"openshift-service-ca.crt\""
Jul 28 20:50:39.001750 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.001624    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-cc370c\"/\"default-dockercfg-pgvmh\""
Jul 28 20:50:39.001750 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.001648    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-cc370c\"/\"kube-root-ca.crt\""
Jul 28 20:50:39.011934 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.011908    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"]
Jul 28 20:50:39.128864 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.128822    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-model-cache\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.129058 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.128885    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-home\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.129058 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.128970    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdzz8\" (UniqueName: \"kubernetes.io/projected/b44e339d-58be-48ef-b520-a591bb84ec9f-kube-api-access-bdzz8\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.129161 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.129070    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-dshm\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.129161 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.129100    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b44e339d-58be-48ef-b520-a591bb84ec9f-tls-certs\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.129161 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.129147    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-tmp-dir\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.229706 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.229666    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-dshm\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.229706 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.229707    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b44e339d-58be-48ef-b520-a591bb84ec9f-tls-certs\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.229983 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.229741    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-tmp-dir\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.229983 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.229761    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-model-cache\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.229983 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.229802    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-home\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.229983 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.229840    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdzz8\" (UniqueName: \"kubernetes.io/projected/b44e339d-58be-48ef-b520-a591bb84ec9f-kube-api-access-bdzz8\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.230216 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.230192    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-tmp-dir\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.230280 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.230229    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-home\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.230334 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.230283    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-model-cache\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.232020 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.231995    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-dshm\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.232291 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.232270    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b44e339d-58be-48ef-b520-a591bb84ec9f-tls-certs\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.238820 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.238799    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdzz8\" (UniqueName: \"kubernetes.io/projected/b44e339d-58be-48ef-b520-a591bb84ec9f-kube-api-access-bdzz8\") pod \"auth-test-model-cc370c-kserve-c897dc9f5-8qn4f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") " pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.307614 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.307585    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:39.447028 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:39.446998    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"]
Jul 28 20:50:39.448229 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:50:39.448198    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb44e339d_58be_48ef_b520_a591bb84ec9f.slice/crio-f5bfce475954c35e823c1051356544d8fc710bb37bcc4f4a2fd6ecfd1165e15d WatchSource:0}: Error finding container f5bfce475954c35e823c1051356544d8fc710bb37bcc4f4a2fd6ecfd1165e15d: Status 404 returned error can't find the container with id f5bfce475954c35e823c1051356544d8fc710bb37bcc4f4a2fd6ecfd1165e15d
Jul 28 20:50:40.175232 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:40.175189    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f" event={"ID":"b44e339d-58be-48ef-b520-a591bb84ec9f","Type":"ContainerStarted","Data":"9418f5b824b82a6399149282132c5a30f9354910eba41b7d82e23261f1ad9d3f"}
Jul 28 20:50:40.175232 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:40.175235    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f" event={"ID":"b44e339d-58be-48ef-b520-a591bb84ec9f","Type":"ContainerStarted","Data":"f5bfce475954c35e823c1051356544d8fc710bb37bcc4f4a2fd6ecfd1165e15d"}
Jul 28 20:50:40.194406 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:40.194346    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f" podStartSLOduration=2.194327972 podStartE2EDuration="2.194327972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:50:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:50:40.193093122 +0000 UTC m=+2517.435186341" watchObservedRunningTime="2026-07-28 20:50:40.194327972 +0000 UTC m=+2517.436421194"
Jul 28 20:50:41.996029 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:41.995990    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:50:49.308299 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:49.308250    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:49.308299 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:49.308318    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:49.321000 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:49.320967    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:50:50.241034 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:50:50.240998    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:51:14.487761 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:51:14.487723    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:51:40.891468 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:51:40.891428    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:52:01.490786 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:52:01.490747    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:52:19.398482 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:52:19.398381    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:52:47.205279 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:52:47.205236    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:53:05.099122 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:53:05.099082    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:53:32.502873 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:53:32.502832    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:53:43.425854 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:53:43.425783    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:53:43.433668 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:53:43.433645    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:53:50.108626 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:53:50.108592    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:54:16.592608 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:54:16.592577    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:54:34.295515 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:54:34.295479    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:54:56.989992 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:54:56.989957    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:55:14.993906 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:55:14.993803    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:55:42.391960 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:55:42.391921    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:55:43.511958 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:55:43.511929    2558 scope.go:117] "RemoveContainer" containerID="4077ab3e3572653059dfbab9b98b24a9518192980cf03239efec38ab1094bd36"
Jul 28 20:55:43.519611 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:55:43.519587    2558 scope.go:117] "RemoveContainer" containerID="cb8b197ec90c7096cb7d90d91cda0f2fcf4b6290003ad00f96ebf547994fada2"
Jul 28 20:55:45.391576 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:55:45.391534    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"]
Jul 28 20:55:58.895087 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:55:58.895052    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:56:00.410277 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.410240    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f" podUID="b44e339d-58be-48ef-b520-a591bb84ec9f" containerName="main" containerID="cri-o://9418f5b824b82a6399149282132c5a30f9354910eba41b7d82e23261f1ad9d3f" gracePeriod=45
Jul 28 20:56:00.653603 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.653576    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:56:00.717296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.717224    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-model-cache\") pod \"b44e339d-58be-48ef-b520-a591bb84ec9f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") "
Jul 28 20:56:00.717296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.717258    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-home\") pod \"b44e339d-58be-48ef-b520-a591bb84ec9f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") "
Jul 28 20:56:00.717296 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.717295    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-tmp-dir\") pod \"b44e339d-58be-48ef-b520-a591bb84ec9f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") "
Jul 28 20:56:00.717576 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.717391    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-dshm\") pod \"b44e339d-58be-48ef-b520-a591bb84ec9f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") "
Jul 28 20:56:00.717576 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.717419    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdzz8\" (UniqueName: \"kubernetes.io/projected/b44e339d-58be-48ef-b520-a591bb84ec9f-kube-api-access-bdzz8\") pod \"b44e339d-58be-48ef-b520-a591bb84ec9f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") "
Jul 28 20:56:00.717576 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.717450    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b44e339d-58be-48ef-b520-a591bb84ec9f-tls-certs\") pod \"b44e339d-58be-48ef-b520-a591bb84ec9f\" (UID: \"b44e339d-58be-48ef-b520-a591bb84ec9f\") "
Jul 28 20:56:00.717576 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.717547    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-home" (OuterVolumeSpecName: "home") pod "b44e339d-58be-48ef-b520-a591bb84ec9f" (UID: "b44e339d-58be-48ef-b520-a591bb84ec9f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:56:00.717747 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.717530    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-model-cache" (OuterVolumeSpecName: "model-cache") pod "b44e339d-58be-48ef-b520-a591bb84ec9f" (UID: "b44e339d-58be-48ef-b520-a591bb84ec9f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:56:00.717747 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.717587    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b44e339d-58be-48ef-b520-a591bb84ec9f" (UID: "b44e339d-58be-48ef-b520-a591bb84ec9f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:56:00.717813 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.717774    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-model-cache\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:56:00.717813 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.717805    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-home\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:56:00.717879 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.717818    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-tmp-dir\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:56:00.719638 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.719608    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-dshm" (OuterVolumeSpecName: "dshm") pod "b44e339d-58be-48ef-b520-a591bb84ec9f" (UID: "b44e339d-58be-48ef-b520-a591bb84ec9f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:56:00.719889 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.719657    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b44e339d-58be-48ef-b520-a591bb84ec9f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b44e339d-58be-48ef-b520-a591bb84ec9f" (UID: "b44e339d-58be-48ef-b520-a591bb84ec9f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:56:00.719889 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.719683    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b44e339d-58be-48ef-b520-a591bb84ec9f-kube-api-access-bdzz8" (OuterVolumeSpecName: "kube-api-access-bdzz8") pod "b44e339d-58be-48ef-b520-a591bb84ec9f" (UID: "b44e339d-58be-48ef-b520-a591bb84ec9f"). InnerVolumeSpecName "kube-api-access-bdzz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:56:00.793508 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.793479    2558 generic.go:356] "Generic (PLEG): container finished" podID="b44e339d-58be-48ef-b520-a591bb84ec9f" containerID="9418f5b824b82a6399149282132c5a30f9354910eba41b7d82e23261f1ad9d3f" exitCode=0
Jul 28 20:56:00.793643 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.793508    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f" event={"ID":"b44e339d-58be-48ef-b520-a591bb84ec9f","Type":"ContainerDied","Data":"9418f5b824b82a6399149282132c5a30f9354910eba41b7d82e23261f1ad9d3f"}
Jul 28 20:56:00.793643 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.793526    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"
Jul 28 20:56:00.793643 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.793549    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f" event={"ID":"b44e339d-58be-48ef-b520-a591bb84ec9f","Type":"ContainerDied","Data":"f5bfce475954c35e823c1051356544d8fc710bb37bcc4f4a2fd6ecfd1165e15d"}
Jul 28 20:56:00.793643 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.793579    2558 scope.go:117] "RemoveContainer" containerID="9418f5b824b82a6399149282132c5a30f9354910eba41b7d82e23261f1ad9d3f"
Jul 28 20:56:00.802173 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.802156    2558 scope.go:117] "RemoveContainer" containerID="9418f5b824b82a6399149282132c5a30f9354910eba41b7d82e23261f1ad9d3f"
Jul 28 20:56:00.802421 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:56:00.802404    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9418f5b824b82a6399149282132c5a30f9354910eba41b7d82e23261f1ad9d3f\": container with ID starting with 9418f5b824b82a6399149282132c5a30f9354910eba41b7d82e23261f1ad9d3f not found: ID does not exist" containerID="9418f5b824b82a6399149282132c5a30f9354910eba41b7d82e23261f1ad9d3f"
Jul 28 20:56:00.802477 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.802431    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9418f5b824b82a6399149282132c5a30f9354910eba41b7d82e23261f1ad9d3f"} err="failed to get container status \"9418f5b824b82a6399149282132c5a30f9354910eba41b7d82e23261f1ad9d3f\": rpc error: code = NotFound desc = could not find container \"9418f5b824b82a6399149282132c5a30f9354910eba41b7d82e23261f1ad9d3f\": container with ID starting with 9418f5b824b82a6399149282132c5a30f9354910eba41b7d82e23261f1ad9d3f not found: ID does not exist"
Jul 28 20:56:00.814691 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.814672    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"]
Jul 28 20:56:00.818098 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.818079    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/auth-test-model-cc370c-kserve-c897dc9f5-8qn4f"]
Jul 28 20:56:00.818438 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.818421    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b44e339d-58be-48ef-b520-a591bb84ec9f-dshm\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:56:00.818491 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.818442    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdzz8\" (UniqueName: \"kubernetes.io/projected/b44e339d-58be-48ef-b520-a591bb84ec9f-kube-api-access-bdzz8\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:56:00.818491 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:00.818453    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b44e339d-58be-48ef-b520-a591bb84ec9f-tls-certs\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:56:01.340458 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:01.340412    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b44e339d-58be-48ef-b520-a591bb84ec9f" path="/var/lib/kubelet/pods/b44e339d-58be-48ef-b520-a591bb84ec9f/volumes"
Jul 28 20:56:03.475681 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.475646    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"]
Jul 28 20:56:03.476114 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.476096    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b44e339d-58be-48ef-b520-a591bb84ec9f" containerName="main"
Jul 28 20:56:03.476114 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.476114    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b44e339d-58be-48ef-b520-a591bb84ec9f" containerName="main"
Jul 28 20:56:03.476236 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.476193    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b44e339d-58be-48ef-b520-a591bb84ec9f" containerName="main"
Jul 28 20:56:03.480550 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.480531    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.483924 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.483897    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-cc370c\"/\"default-dockercfg-pgvmh\""
Jul 28 20:56:03.484051 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.483959    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-cc370c\"/\"kube-root-ca.crt\""
Jul 28 20:56:03.484163 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.484144    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-cc370c\"/\"openshift-service-ca.crt\""
Jul 28 20:56:03.484322 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.484302    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-cc370c\"/\"sub-test-model-cc370c-kserve-self-signed-certs\""
Jul 28 20:56:03.486597 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.486577    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"]
Jul 28 20:56:03.540306 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.540277    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-tmp-dir\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.540480 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.540331    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f47a0fc0-2cf4-4772-86f2-19950e7b1505-tls-certs\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.540480 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.540387    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-dshm\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.540480 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.540432    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-home\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.540627 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.540506    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wx4f\" (UniqueName: \"kubernetes.io/projected/f47a0fc0-2cf4-4772-86f2-19950e7b1505-kube-api-access-9wx4f\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.540627 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.540611    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-model-cache\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.641297 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.641263    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-tmp-dir\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.641488 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.641315    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f47a0fc0-2cf4-4772-86f2-19950e7b1505-tls-certs\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.641488 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.641348    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-dshm\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.641488 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.641395    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-home\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.641488 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.641444    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wx4f\" (UniqueName: \"kubernetes.io/projected/f47a0fc0-2cf4-4772-86f2-19950e7b1505-kube-api-access-9wx4f\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.641701 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.641504    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-model-cache\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.641774 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.641752    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-tmp-dir\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.641834 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.641811    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-home\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.641894 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.641836    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-model-cache\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.643665 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.643641    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-dshm\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.643921 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.643903    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f47a0fc0-2cf4-4772-86f2-19950e7b1505-tls-certs\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.649067 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.649045    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wx4f\" (UniqueName: \"kubernetes.io/projected/f47a0fc0-2cf4-4772-86f2-19950e7b1505-kube-api-access-9wx4f\") pod \"sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") " pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.791790 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.791765    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:03.943337 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:03.943310    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"]
Jul 28 20:56:03.946032 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:56:03.946006    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf47a0fc0_2cf4_4772_86f2_19950e7b1505.slice/crio-0c5c04524822c29427c4f44017d0dc2c2817a68219a04de3d3604106a901474f WatchSource:0}: Error finding container 0c5c04524822c29427c4f44017d0dc2c2817a68219a04de3d3604106a901474f: Status 404 returned error can't find the container with id 0c5c04524822c29427c4f44017d0dc2c2817a68219a04de3d3604106a901474f
Jul 28 20:56:04.825161 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:04.825120    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp" event={"ID":"f47a0fc0-2cf4-4772-86f2-19950e7b1505","Type":"ContainerStarted","Data":"b3e3c46f878db68519f924d9b3f6e4dc5db750a729679b9c1e448ddb5c09814f"}
Jul 28 20:56:04.825161 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:04.825166    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp" event={"ID":"f47a0fc0-2cf4-4772-86f2-19950e7b1505","Type":"ContainerStarted","Data":"0c5c04524822c29427c4f44017d0dc2c2817a68219a04de3d3604106a901474f"}
Jul 28 20:56:04.844180 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:04.844129    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp" podStartSLOduration=1.8441106600000001 podStartE2EDuration="1.84411066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:56:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:56:04.842387369 +0000 UTC m=+2842.084480593" watchObservedRunningTime="2026-07-28 20:56:04.84411066 +0000 UTC m=+2842.086203920"
Jul 28 20:56:06.089716 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:06.089680    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:56:13.792412 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:13.792376    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:13.792906 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:13.792436    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:13.804771 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:13.804749    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:13.879172 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:13.879146    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:56:35.799971 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:56:35.799880    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:57:40.789664 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:57:40.789632    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:57:59.489694 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:57:59.489654    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:58:06.512519 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:06.512422    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"]
Jul 28 20:58:18.804021 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.803981    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"]
Jul 28 20:58:18.807592 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.807573    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.810039 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.810016    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-cc370c\"/\"rate-test-model-cc370c-kserve-self-signed-certs\""
Jul 28 20:58:18.817551 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.817527    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"]
Jul 28 20:58:18.895998 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.895973    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-model-cache\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.896114 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.896004    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-tmp-dir\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.896114 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.896034    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-dshm\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.896114 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.896055    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-home\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.896232 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.896114    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-tls-certs\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.896232 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.896155    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7pl9\" (UniqueName: \"kubernetes.io/projected/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-kube-api-access-k7pl9\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.980229 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.980204    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:58:18.996842 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.996820    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-model-cache\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.996927 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.996853    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-tmp-dir\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.996927 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.996888    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-dshm\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.997015 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.996994    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-home\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.997053 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.997042    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-tls-certs\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.997110 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.997086    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7pl9\" (UniqueName: \"kubernetes.io/projected/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-kube-api-access-k7pl9\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.997269 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.997247    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-model-cache\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.997345 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.997281    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-tmp-dir\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.997345 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.997322    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-home\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.999085 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.999058    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-dshm\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:18.999519 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:18.999497    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-tls-certs\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:19.004047 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:19.004029    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7pl9\" (UniqueName: \"kubernetes.io/projected/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-kube-api-access-k7pl9\") pod \"rate-test-model-cc370c-kserve-6b78847f95-p8vsd\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") " pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:19.117636 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:19.117580    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:19.249879 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:19.249852    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"]
Jul 28 20:58:19.251384 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:58:19.251359    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd238da7_bf12_40eb_aeb1_fa8b8e80a9ac.slice/crio-1ae4e068203319bcaa6def5c4627752333d21b5db441893b7beecbdbc02e059f WatchSource:0}: Error finding container 1ae4e068203319bcaa6def5c4627752333d21b5db441893b7beecbdbc02e059f: Status 404 returned error can't find the container with id 1ae4e068203319bcaa6def5c4627752333d21b5db441893b7beecbdbc02e059f
Jul 28 20:58:19.484756 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:19.484660    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd" event={"ID":"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac","Type":"ContainerStarted","Data":"91dde08e4c806922fcae5b3f713051856c350165bd7d2a7d7808cc26e209ba4c"}
Jul 28 20:58:19.484756 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:19.484700    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd" event={"ID":"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac","Type":"ContainerStarted","Data":"1ae4e068203319bcaa6def5c4627752333d21b5db441893b7beecbdbc02e059f"}
Jul 28 20:58:19.501868 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:19.501821    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd" podStartSLOduration=1.5018046360000001 podStartE2EDuration="1.501804636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:58:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:58:19.500739941 +0000 UTC m=+2976.742833165" watchObservedRunningTime="2026-07-28 20:58:19.501804636 +0000 UTC m=+2976.743897886"
Jul 28 20:58:21.531512 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.531472    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp" podUID="f47a0fc0-2cf4-4772-86f2-19950e7b1505" containerName="main" containerID="cri-o://b3e3c46f878db68519f924d9b3f6e4dc5db750a729679b9c1e448ddb5c09814f" gracePeriod=45
Jul 28 20:58:21.784167 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.784107    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:58:21.925190 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.925163    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wx4f\" (UniqueName: \"kubernetes.io/projected/f47a0fc0-2cf4-4772-86f2-19950e7b1505-kube-api-access-9wx4f\") pod \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") "
Jul 28 20:58:21.925320 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.925249    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-model-cache\") pod \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") "
Jul 28 20:58:21.925372 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.925353    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-dshm\") pod \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") "
Jul 28 20:58:21.925414 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.925405    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f47a0fc0-2cf4-4772-86f2-19950e7b1505-tls-certs\") pod \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") "
Jul 28 20:58:21.925450 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.925430    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-tmp-dir\") pod \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") "
Jul 28 20:58:21.925450 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.925437    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-model-cache" (OuterVolumeSpecName: "model-cache") pod "f47a0fc0-2cf4-4772-86f2-19950e7b1505" (UID: "f47a0fc0-2cf4-4772-86f2-19950e7b1505"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:58:21.925530 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.925468    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-home\") pod \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\" (UID: \"f47a0fc0-2cf4-4772-86f2-19950e7b1505\") "
Jul 28 20:58:21.925686 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.925662    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f47a0fc0-2cf4-4772-86f2-19950e7b1505" (UID: "f47a0fc0-2cf4-4772-86f2-19950e7b1505"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:58:21.925765 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.925741    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-home" (OuterVolumeSpecName: "home") pod "f47a0fc0-2cf4-4772-86f2-19950e7b1505" (UID: "f47a0fc0-2cf4-4772-86f2-19950e7b1505"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:58:21.925903 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.925878    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-model-cache\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:58:21.925903 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.925900    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-tmp-dir\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:58:21.926056 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.925913    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-home\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:58:21.927381 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.927350    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-dshm" (OuterVolumeSpecName: "dshm") pod "f47a0fc0-2cf4-4772-86f2-19950e7b1505" (UID: "f47a0fc0-2cf4-4772-86f2-19950e7b1505"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:58:21.927492 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.927430    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f47a0fc0-2cf4-4772-86f2-19950e7b1505-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f47a0fc0-2cf4-4772-86f2-19950e7b1505" (UID: "f47a0fc0-2cf4-4772-86f2-19950e7b1505"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:58:21.927492 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:21.927461    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f47a0fc0-2cf4-4772-86f2-19950e7b1505-kube-api-access-9wx4f" (OuterVolumeSpecName: "kube-api-access-9wx4f") pod "f47a0fc0-2cf4-4772-86f2-19950e7b1505" (UID: "f47a0fc0-2cf4-4772-86f2-19950e7b1505"). InnerVolumeSpecName "kube-api-access-9wx4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:58:22.026714 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:22.026692    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wx4f\" (UniqueName: \"kubernetes.io/projected/f47a0fc0-2cf4-4772-86f2-19950e7b1505-kube-api-access-9wx4f\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:58:22.026714 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:22.026713    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f47a0fc0-2cf4-4772-86f2-19950e7b1505-dshm\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:58:22.026835 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:22.026723    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f47a0fc0-2cf4-4772-86f2-19950e7b1505-tls-certs\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 20:58:22.501527 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:22.501492    2558 generic.go:356] "Generic (PLEG): container finished" podID="f47a0fc0-2cf4-4772-86f2-19950e7b1505" containerID="b3e3c46f878db68519f924d9b3f6e4dc5db750a729679b9c1e448ddb5c09814f" exitCode=0
Jul 28 20:58:22.501722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:22.501537    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"
Jul 28 20:58:22.501722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:22.501595    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp" event={"ID":"f47a0fc0-2cf4-4772-86f2-19950e7b1505","Type":"ContainerDied","Data":"b3e3c46f878db68519f924d9b3f6e4dc5db750a729679b9c1e448ddb5c09814f"}
Jul 28 20:58:22.501722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:22.501637    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp" event={"ID":"f47a0fc0-2cf4-4772-86f2-19950e7b1505","Type":"ContainerDied","Data":"0c5c04524822c29427c4f44017d0dc2c2817a68219a04de3d3604106a901474f"}
Jul 28 20:58:22.501722 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:22.501659    2558 scope.go:117] "RemoveContainer" containerID="b3e3c46f878db68519f924d9b3f6e4dc5db750a729679b9c1e448ddb5c09814f"
Jul 28 20:58:22.510740 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:22.510722    2558 scope.go:117] "RemoveContainer" containerID="b3e3c46f878db68519f924d9b3f6e4dc5db750a729679b9c1e448ddb5c09814f"
Jul 28 20:58:22.510993 ip-10-0-133-72 kubenswrapper[2558]: E0728 20:58:22.510976    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3e3c46f878db68519f924d9b3f6e4dc5db750a729679b9c1e448ddb5c09814f\": container with ID starting with b3e3c46f878db68519f924d9b3f6e4dc5db750a729679b9c1e448ddb5c09814f not found: ID does not exist" containerID="b3e3c46f878db68519f924d9b3f6e4dc5db750a729679b9c1e448ddb5c09814f"
Jul 28 20:58:22.511040 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:22.511002    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e3c46f878db68519f924d9b3f6e4dc5db750a729679b9c1e448ddb5c09814f"} err="failed to get container status \"b3e3c46f878db68519f924d9b3f6e4dc5db750a729679b9c1e448ddb5c09814f\": rpc error: code = NotFound desc = could not find container \"b3e3c46f878db68519f924d9b3f6e4dc5db750a729679b9c1e448ddb5c09814f\": container with ID starting with b3e3c46f878db68519f924d9b3f6e4dc5db750a729679b9c1e448ddb5c09814f not found: ID does not exist"
Jul 28 20:58:22.525063 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:22.525037    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"]
Jul 28 20:58:22.528373 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:22.528349    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/sub-test-model-cc370c-kserve-6b77fbd5cc-vttlp"]
Jul 28 20:58:23.340900 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:23.340868    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f47a0fc0-2cf4-4772-86f2-19950e7b1505" path="/var/lib/kubelet/pods/f47a0fc0-2cf4-4772-86f2-19950e7b1505/volumes"
Jul 28 20:58:28.795913 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:28.795878    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:58:29.118409 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:29.118341    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:29.118409 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:29.118387    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:29.130341 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:29.130316    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:29.554061 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:29.554033    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 20:58:43.456714 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:43.456688    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:58:43.466448 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:58:43.466425    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 20:59:15.292174 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:15.292139    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 20:59:50.051355 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.051274    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f96kr"]
Jul 28 20:59:50.051884 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.051713    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f47a0fc0-2cf4-4772-86f2-19950e7b1505" containerName="main"
Jul 28 20:59:50.051884 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.051725    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f47a0fc0-2cf4-4772-86f2-19950e7b1505" containerName="main"
Jul 28 20:59:50.051884 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.051798    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f47a0fc0-2cf4-4772-86f2-19950e7b1505" containerName="main"
Jul 28 20:59:50.055084 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.055067    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f96kr"
Jul 28 20:59:50.058049 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.058026    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-zm59r\""
Jul 28 20:59:50.064043 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.064020    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f96kr"]
Jul 28 20:59:50.168467 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.168440    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfqp4\" (UniqueName: \"kubernetes.io/projected/708c465c-a8dd-4caa-b964-49403c029fa3-kube-api-access-cfqp4\") pod \"kuadrant-operator-catalog-f96kr\" (UID: \"708c465c-a8dd-4caa-b964-49403c029fa3\") " pod="kuadrant-system/kuadrant-operator-catalog-f96kr"
Jul 28 20:59:50.269183 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.269150    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfqp4\" (UniqueName: \"kubernetes.io/projected/708c465c-a8dd-4caa-b964-49403c029fa3-kube-api-access-cfqp4\") pod \"kuadrant-operator-catalog-f96kr\" (UID: \"708c465c-a8dd-4caa-b964-49403c029fa3\") " pod="kuadrant-system/kuadrant-operator-catalog-f96kr"
Jul 28 20:59:50.277691 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.277670    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfqp4\" (UniqueName: \"kubernetes.io/projected/708c465c-a8dd-4caa-b964-49403c029fa3-kube-api-access-cfqp4\") pod \"kuadrant-operator-catalog-f96kr\" (UID: \"708c465c-a8dd-4caa-b964-49403c029fa3\") " pod="kuadrant-system/kuadrant-operator-catalog-f96kr"
Jul 28 20:59:50.366096 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.366045    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f96kr"
Jul 28 20:59:50.496151 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.496125    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f96kr"]
Jul 28 20:59:50.498575 ip-10-0-133-72 kubenswrapper[2558]: W0728 20:59:50.498526    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod708c465c_a8dd_4caa_b964_49403c029fa3.slice/crio-63d4c1fbd7130451b32b89ceccbb0aa012214335c959254bb2c30287c38b4f67 WatchSource:0}: Error finding container 63d4c1fbd7130451b32b89ceccbb0aa012214335c959254bb2c30287c38b4f67: Status 404 returned error can't find the container with id 63d4c1fbd7130451b32b89ceccbb0aa012214335c959254bb2c30287c38b4f67
Jul 28 20:59:50.499931 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.499914    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:59:50.935621 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:50.935584    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f96kr" event={"ID":"708c465c-a8dd-4caa-b964-49403c029fa3","Type":"ContainerStarted","Data":"63d4c1fbd7130451b32b89ceccbb0aa012214335c959254bb2c30287c38b4f67"}
Jul 28 20:59:52.947708 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:52.947671    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f96kr" event={"ID":"708c465c-a8dd-4caa-b964-49403c029fa3","Type":"ContainerStarted","Data":"aea36fac87febe5b847c6e63de8f030be9719506864fc8738d84107150e28aef"}
Jul 28 20:59:52.963213 ip-10-0-133-72 kubenswrapper[2558]: I0728 20:59:52.963167    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-f96kr" podStartSLOduration=1.34074673 podStartE2EDuration="2.963151851s" totalImagesPullingTime="1.622405121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:59:52.962638701 +0000 UTC m=+3070.204731925" watchObservedRunningTime="2026-07-28 20:59:52.963151851 +0000 UTC m=+3070.205245074"
Jul 28 21:00:00.157888 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.157855    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n"]
Jul 28 21:00:00.161332 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.161316    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx"]
Jul 28 21:00:00.161443 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.161423    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n"
Jul 28 21:00:00.164607 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.164589    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754540-mgfpc"]
Jul 28 21:00:00.164749 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.164729    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx"
Jul 28 21:00:00.165674 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.165659    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gcrkz\""
Jul 28 21:00:00.167717 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.167699    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-mgfpc"
Jul 28 21:00:00.172699 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.172681    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n"
Jul 28 21:00:00.175069 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.175049    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n"]
Jul 28 21:00:00.181627 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.181605    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx"]
Jul 28 21:00:00.183378 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.183362    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx"
Jul 28 21:00:00.183528 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.183510    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754540-mgfpc"]
Jul 28 21:00:00.220777 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.220750    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-mgfpc"
Jul 28 21:00:00.342451 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.342420    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n"]
Jul 28 21:00:00.344035 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:00:00.344002    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b498bab_8b32_4065_9c54_59b53f1739ee.slice/crio-b89fea7e442e53a1f70274426129802953d36cf8f4e0bb8c9212c40a0212d7c6 WatchSource:0}: Error finding container b89fea7e442e53a1f70274426129802953d36cf8f4e0bb8c9212c40a0212d7c6: Status 404 returned error can't find the container with id b89fea7e442e53a1f70274426129802953d36cf8f4e0bb8c9212c40a0212d7c6
Jul 28 21:00:00.366239 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.366217    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-f96kr"
Jul 28 21:00:00.366368 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.366293    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-f96kr"
Jul 28 21:00:00.392253 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.392224    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754540-mgfpc"]
Jul 28 21:00:00.392442 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.392428    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-f96kr"
Jul 28 21:00:00.393176 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:00:00.393154    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb07ba0d_3b85_4ef6_8d42_cd23ba191b74.slice/crio-9454a7420d05133bb9d408a9cae05dd8638c9787538872102a43e3300030026a WatchSource:0}: Error finding container 9454a7420d05133bb9d408a9cae05dd8638c9787538872102a43e3300030026a: Status 404 returned error can't find the container with id 9454a7420d05133bb9d408a9cae05dd8638c9787538872102a43e3300030026a
Jul 28 21:00:00.566497 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.566467    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx"]
Jul 28 21:00:00.622852 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:00:00.622793    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3de6d1d2_fed2_43f4_bcfd_4c5dd118260f.slice/crio-8db9c5dad2e8a163e67940a7534ab541b4a7e684a74545ae61566c432e03159f WatchSource:0}: Error finding container 8db9c5dad2e8a163e67940a7534ab541b4a7e684a74545ae61566c432e03159f: Status 404 returned error can't find the container with id 8db9c5dad2e8a163e67940a7534ab541b4a7e684a74545ae61566c432e03159f
Jul 28 21:00:00.991455 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.991387    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx" event={"ID":"3de6d1d2-fed2-43f4-bcfd-4c5dd118260f","Type":"ContainerStarted","Data":"15a31fa1b3c4b0ab3e85168ae586a203d2cbf176865a76340c23aff4226e829c"}
Jul 28 21:00:00.991455 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.991448    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx" event={"ID":"3de6d1d2-fed2-43f4-bcfd-4c5dd118260f","Type":"ContainerStarted","Data":"8db9c5dad2e8a163e67940a7534ab541b4a7e684a74545ae61566c432e03159f"}
Jul 28 21:00:00.992801 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.992773    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n" event={"ID":"5b498bab-8b32-4065-9c54-59b53f1739ee","Type":"ContainerStarted","Data":"78bd7e2e983d3bfe27a531fe1291ec6095b5f44dcfcbf49866410cb172c17b31"}
Jul 28 21:00:00.992911 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.992823    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n" event={"ID":"5b498bab-8b32-4065-9c54-59b53f1739ee","Type":"ContainerStarted","Data":"b89fea7e442e53a1f70274426129802953d36cf8f4e0bb8c9212c40a0212d7c6"}
Jul 28 21:00:00.994224 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.994202    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-mgfpc" event={"ID":"bb07ba0d-3b85-4ef6-8d42-cd23ba191b74","Type":"ContainerStarted","Data":"0e5fc8c36d0fe778402132a2b134fbbb5d6f78f00aed880fd46337c532810f6c"}
Jul 28 21:00:00.994305 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:00.994246    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-mgfpc" event={"ID":"bb07ba0d-3b85-4ef6-8d42-cd23ba191b74","Type":"ContainerStarted","Data":"9454a7420d05133bb9d408a9cae05dd8638c9787538872102a43e3300030026a"}
Jul 28 21:00:01.015417 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:01.015397    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-f96kr"
Jul 28 21:00:01.033382 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:01.033348    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx" podStartSLOduration=1.033335604 podStartE2EDuration="1.033335604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:00:01.033199603 +0000 UTC m=+3078.275292826" watchObservedRunningTime="2026-07-28 21:00:01.033335604 +0000 UTC m=+3078.275428809"
Jul 28 21:00:01.066865 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:01.066820    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-mgfpc" podStartSLOduration=1.066806413 podStartE2EDuration="1.066806413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:00:01.065297099 +0000 UTC m=+3078.307390322" watchObservedRunningTime="2026-07-28 21:00:01.066806413 +0000 UTC m=+3078.308899635"
Jul 28 21:00:01.074899 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:01.074877    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f96kr"]
Jul 28 21:00:01.087764 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:01.087727    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n" podStartSLOduration=1.087715221 podStartE2EDuration="1.087715221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:00:01.086936667 +0000 UTC m=+3078.329029889" watchObservedRunningTime="2026-07-28 21:00:01.087715221 +0000 UTC m=+3078.329808421"
Jul 28 21:00:02.000654 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:02.000613    2558 generic.go:356] "Generic (PLEG): container finished" podID="bb07ba0d-3b85-4ef6-8d42-cd23ba191b74" containerID="0e5fc8c36d0fe778402132a2b134fbbb5d6f78f00aed880fd46337c532810f6c" exitCode=0
Jul 28 21:00:02.001117 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:02.000708    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-mgfpc" event={"ID":"bb07ba0d-3b85-4ef6-8d42-cd23ba191b74","Type":"ContainerDied","Data":"0e5fc8c36d0fe778402132a2b134fbbb5d6f78f00aed880fd46337c532810f6c"}
Jul 28 21:00:02.002302 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:02.002277    2558 generic.go:356] "Generic (PLEG): container finished" podID="3de6d1d2-fed2-43f4-bcfd-4c5dd118260f" containerID="15a31fa1b3c4b0ab3e85168ae586a203d2cbf176865a76340c23aff4226e829c" exitCode=0
Jul 28 21:00:02.002423 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:02.002359    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx" event={"ID":"3de6d1d2-fed2-43f4-bcfd-4c5dd118260f","Type":"ContainerDied","Data":"15a31fa1b3c4b0ab3e85168ae586a203d2cbf176865a76340c23aff4226e829c"}
Jul 28 21:00:02.003782 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:02.003760    2558 generic.go:356] "Generic (PLEG): container finished" podID="5b498bab-8b32-4065-9c54-59b53f1739ee" containerID="78bd7e2e983d3bfe27a531fe1291ec6095b5f44dcfcbf49866410cb172c17b31" exitCode=0
Jul 28 21:00:02.003867 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:02.003824    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n" event={"ID":"5b498bab-8b32-4065-9c54-59b53f1739ee","Type":"ContainerDied","Data":"78bd7e2e983d3bfe27a531fe1291ec6095b5f44dcfcbf49866410cb172c17b31"}
Jul 28 21:00:03.008837 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:03.008799    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-f96kr" podUID="708c465c-a8dd-4caa-b964-49403c029fa3" containerName="registry-server" containerID="cri-o://aea36fac87febe5b847c6e63de8f030be9719506864fc8738d84107150e28aef" gracePeriod=2
Jul 28 21:00:03.184976 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:03.184954    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx"
Jul 28 21:00:03.188945 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:03.188927    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n"
Jul 28 21:00:03.193113 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:03.193092    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-mgfpc"
Jul 28 21:00:03.257642 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:03.257614    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f96kr"
Jul 28 21:00:03.389469 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:03.389442    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfqp4\" (UniqueName: \"kubernetes.io/projected/708c465c-a8dd-4caa-b964-49403c029fa3-kube-api-access-cfqp4\") pod \"708c465c-a8dd-4caa-b964-49403c029fa3\" (UID: \"708c465c-a8dd-4caa-b964-49403c029fa3\") "
Jul 28 21:00:03.391421 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:03.391399    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/708c465c-a8dd-4caa-b964-49403c029fa3-kube-api-access-cfqp4" (OuterVolumeSpecName: "kube-api-access-cfqp4") pod "708c465c-a8dd-4caa-b964-49403c029fa3" (UID: "708c465c-a8dd-4caa-b964-49403c029fa3"). InnerVolumeSpecName "kube-api-access-cfqp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:00:03.490425 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:03.490401    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfqp4\" (UniqueName: \"kubernetes.io/projected/708c465c-a8dd-4caa-b964-49403c029fa3-kube-api-access-cfqp4\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:00:04.015311 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.015273    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx"
Jul 28 21:00:04.015311 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.015300    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx" event={"ID":"3de6d1d2-fed2-43f4-bcfd-4c5dd118260f","Type":"ContainerDied","Data":"8db9c5dad2e8a163e67940a7534ab541b4a7e684a74545ae61566c432e03159f"}
Jul 28 21:00:04.015784 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.015351    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8db9c5dad2e8a163e67940a7534ab541b4a7e684a74545ae61566c432e03159f"
Jul 28 21:00:04.016772 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.016748    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n"
Jul 28 21:00:04.016772 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.016762    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n" event={"ID":"5b498bab-8b32-4065-9c54-59b53f1739ee","Type":"ContainerDied","Data":"b89fea7e442e53a1f70274426129802953d36cf8f4e0bb8c9212c40a0212d7c6"}
Jul 28 21:00:04.016967 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.016803    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b89fea7e442e53a1f70274426129802953d36cf8f4e0bb8c9212c40a0212d7c6"
Jul 28 21:00:04.018272 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.018249    2558 generic.go:356] "Generic (PLEG): container finished" podID="708c465c-a8dd-4caa-b964-49403c029fa3" containerID="aea36fac87febe5b847c6e63de8f030be9719506864fc8738d84107150e28aef" exitCode=0
Jul 28 21:00:04.018379 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.018338    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f96kr" event={"ID":"708c465c-a8dd-4caa-b964-49403c029fa3","Type":"ContainerDied","Data":"aea36fac87febe5b847c6e63de8f030be9719506864fc8738d84107150e28aef"}
Jul 28 21:00:04.018379 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.018352    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f96kr"
Jul 28 21:00:04.018379 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.018374    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f96kr" event={"ID":"708c465c-a8dd-4caa-b964-49403c029fa3","Type":"ContainerDied","Data":"63d4c1fbd7130451b32b89ceccbb0aa012214335c959254bb2c30287c38b4f67"}
Jul 28 21:00:04.018766 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.018404    2558 scope.go:117] "RemoveContainer" containerID="aea36fac87febe5b847c6e63de8f030be9719506864fc8738d84107150e28aef"
Jul 28 21:00:04.020010 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.019986    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-mgfpc" event={"ID":"bb07ba0d-3b85-4ef6-8d42-cd23ba191b74","Type":"ContainerDied","Data":"9454a7420d05133bb9d408a9cae05dd8638c9787538872102a43e3300030026a"}
Jul 28 21:00:04.020087 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.020015    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-mgfpc"
Jul 28 21:00:04.020087 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.020030    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9454a7420d05133bb9d408a9cae05dd8638c9787538872102a43e3300030026a"
Jul 28 21:00:04.038611 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.038584    2558 scope.go:117] "RemoveContainer" containerID="aea36fac87febe5b847c6e63de8f030be9719506864fc8738d84107150e28aef"
Jul 28 21:00:04.038958 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:00:04.038932    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aea36fac87febe5b847c6e63de8f030be9719506864fc8738d84107150e28aef\": container with ID starting with aea36fac87febe5b847c6e63de8f030be9719506864fc8738d84107150e28aef not found: ID does not exist" containerID="aea36fac87febe5b847c6e63de8f030be9719506864fc8738d84107150e28aef"
Jul 28 21:00:04.039044 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.038966    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aea36fac87febe5b847c6e63de8f030be9719506864fc8738d84107150e28aef"} err="failed to get container status \"aea36fac87febe5b847c6e63de8f030be9719506864fc8738d84107150e28aef\": rpc error: code = NotFound desc = could not find container \"aea36fac87febe5b847c6e63de8f030be9719506864fc8738d84107150e28aef\": container with ID starting with aea36fac87febe5b847c6e63de8f030be9719506864fc8738d84107150e28aef not found: ID does not exist"
Jul 28 21:00:04.059746 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.059720    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f96kr"]
Jul 28 21:00:04.062427 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:04.062405    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f96kr"]
Jul 28 21:00:05.340899 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:05.340862    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="708c465c-a8dd-4caa-b964-49403c029fa3" path="/var/lib/kubelet/pods/708c465c-a8dd-4caa-b964-49403c029fa3/volumes"
Jul 28 21:00:12.492287 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:12.492255    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:00:32.034756 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:32.034717    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:00:39.579998 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:39.579965    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"]
Jul 28 21:00:42.614548 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.614516    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7572-139675858378416"]
Jul 28 21:00:42.616830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.614895    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="708c465c-a8dd-4caa-b964-49403c029fa3" containerName="registry-server"
Jul 28 21:00:42.616830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.614907    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="708c465c-a8dd-4caa-b964-49403c029fa3" containerName="registry-server"
Jul 28 21:00:42.616830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.614918    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3de6d1d2-fed2-43f4-bcfd-4c5dd118260f" containerName="cleanup"
Jul 28 21:00:42.616830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.614924    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3de6d1d2-fed2-43f4-bcfd-4c5dd118260f" containerName="cleanup"
Jul 28 21:00:42.616830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.614935    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b498bab-8b32-4065-9c54-59b53f1739ee" containerName="cleanup"
Jul 28 21:00:42.616830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.614940    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b498bab-8b32-4065-9c54-59b53f1739ee" containerName="cleanup"
Jul 28 21:00:42.616830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.614947    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb07ba0d-3b85-4ef6-8d42-cd23ba191b74" containerName="cleanup"
Jul 28 21:00:42.616830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.614952    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb07ba0d-3b85-4ef6-8d42-cd23ba191b74" containerName="cleanup"
Jul 28 21:00:42.616830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.615015    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="708c465c-a8dd-4caa-b964-49403c029fa3" containerName="registry-server"
Jul 28 21:00:42.616830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.615023    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb07ba0d-3b85-4ef6-8d42-cd23ba191b74" containerName="cleanup"
Jul 28 21:00:42.616830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.615031    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3de6d1d2-fed2-43f4-bcfd-4c5dd118260f" containerName="cleanup"
Jul 28 21:00:42.616830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.615038    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b498bab-8b32-4065-9c54-59b53f1739ee" containerName="cleanup"
Jul 28 21:00:42.617896 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.617876    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858378416"
Jul 28 21:00:42.623342 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.623320    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cqrvn\""
Jul 28 21:00:42.632647 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.632624    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7572-139675858378416"]
Jul 28 21:00:42.726326 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.726296    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5f4f\" (UniqueName: \"kubernetes.io/projected/39efa226-17d2-46d4-b3b2-be61ea2150bf-kube-api-access-x5f4f\") pod \"test-curl-7572-139675858378416\" (UID: \"39efa226-17d2-46d4-b3b2-be61ea2150bf\") " pod="opendatahub/test-curl-7572-139675858378416"
Jul 28 21:00:42.827144 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.827102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5f4f\" (UniqueName: \"kubernetes.io/projected/39efa226-17d2-46d4-b3b2-be61ea2150bf-kube-api-access-x5f4f\") pod \"test-curl-7572-139675858378416\" (UID: \"39efa226-17d2-46d4-b3b2-be61ea2150bf\") " pod="opendatahub/test-curl-7572-139675858378416"
Jul 28 21:00:42.840467 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.840440    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5f4f\" (UniqueName: \"kubernetes.io/projected/39efa226-17d2-46d4-b3b2-be61ea2150bf-kube-api-access-x5f4f\") pod \"test-curl-7572-139675858378416\" (UID: \"39efa226-17d2-46d4-b3b2-be61ea2150bf\") " pod="opendatahub/test-curl-7572-139675858378416"
Jul 28 21:00:42.927771 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:42.927676    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858378416"
Jul 28 21:00:43.059917 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:43.059884    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7572-139675858378416"]
Jul 28 21:00:43.061252 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:00:43.061229    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39efa226_17d2_46d4_b3b2_be61ea2150bf.slice/crio-7b00dc143dd02c53ba61db257c870ffc3d17c69767d6e19f3ccd04bc47e5f374 WatchSource:0}: Error finding container 7b00dc143dd02c53ba61db257c870ffc3d17c69767d6e19f3ccd04bc47e5f374: Status 404 returned error can't find the container with id 7b00dc143dd02c53ba61db257c870ffc3d17c69767d6e19f3ccd04bc47e5f374
Jul 28 21:00:43.223295 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:43.223229    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7572-139675858378416" event={"ID":"39efa226-17d2-46d4-b3b2-be61ea2150bf","Type":"ContainerStarted","Data":"7b00dc143dd02c53ba61db257c870ffc3d17c69767d6e19f3ccd04bc47e5f374"}
Jul 28 21:00:45.234644 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:45.234602    2558 generic.go:356] "Generic (PLEG): container finished" podID="39efa226-17d2-46d4-b3b2-be61ea2150bf" containerID="15263667bf3072183eea689aed4adcb31d64b62a7d92d4331355226344afe0ee" exitCode=0
Jul 28 21:00:45.235013 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:45.234688    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7572-139675858378416" event={"ID":"39efa226-17d2-46d4-b3b2-be61ea2150bf","Type":"ContainerDied","Data":"15263667bf3072183eea689aed4adcb31d64b62a7d92d4331355226344afe0ee"}
Jul 28 21:00:46.380421 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.380393    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858378416"
Jul 28 21:00:46.395432 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.395408    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:00:46.431195 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.431175    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7572-139675858378416_39efa226-17d2-46d4-b3b2-be61ea2150bf/test-curl-7572-139675858378416/0.log"
Jul 28 21:00:46.459125 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.459101    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7572-139675858378416"]
Jul 28 21:00:46.462199 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.462173    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7572-139675858378416"]
Jul 28 21:00:46.462421 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.462402    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5f4f\" (UniqueName: \"kubernetes.io/projected/39efa226-17d2-46d4-b3b2-be61ea2150bf-kube-api-access-x5f4f\") pod \"39efa226-17d2-46d4-b3b2-be61ea2150bf\" (UID: \"39efa226-17d2-46d4-b3b2-be61ea2150bf\") "
Jul 28 21:00:46.464658 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.464636    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39efa226-17d2-46d4-b3b2-be61ea2150bf-kube-api-access-x5f4f" (OuterVolumeSpecName: "kube-api-access-x5f4f") pod "39efa226-17d2-46d4-b3b2-be61ea2150bf" (UID: "39efa226-17d2-46d4-b3b2-be61ea2150bf"). InnerVolumeSpecName "kube-api-access-x5f4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:00:46.563808 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.563783    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5f4f\" (UniqueName: \"kubernetes.io/projected/39efa226-17d2-46d4-b3b2-be61ea2150bf-kube-api-access-x5f4f\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:00:46.597448 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.597422    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7572-139675858425648"]
Jul 28 21:00:46.597849 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.597835    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39efa226-17d2-46d4-b3b2-be61ea2150bf" containerName="test-curl-7572-139675858378416"
Jul 28 21:00:46.597894 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.597851    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="39efa226-17d2-46d4-b3b2-be61ea2150bf" containerName="test-curl-7572-139675858378416"
Jul 28 21:00:46.597940 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.597931    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="39efa226-17d2-46d4-b3b2-be61ea2150bf" containerName="test-curl-7572-139675858378416"
Jul 28 21:00:46.601042 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.601027    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858425648"
Jul 28 21:00:46.608574 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.608532    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7572-139675858425648"]
Jul 28 21:00:46.764913 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.764887    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbs9z\" (UniqueName: \"kubernetes.io/projected/5e825ce1-2ad9-4309-a65f-620303b66e0e-kube-api-access-rbs9z\") pod \"test-curl-7572-139675858425648\" (UID: \"5e825ce1-2ad9-4309-a65f-620303b66e0e\") " pod="opendatahub/test-curl-7572-139675858425648"
Jul 28 21:00:46.866327 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.866250    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbs9z\" (UniqueName: \"kubernetes.io/projected/5e825ce1-2ad9-4309-a65f-620303b66e0e-kube-api-access-rbs9z\") pod \"test-curl-7572-139675858425648\" (UID: \"5e825ce1-2ad9-4309-a65f-620303b66e0e\") " pod="opendatahub/test-curl-7572-139675858425648"
Jul 28 21:00:46.875306 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.875276    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbs9z\" (UniqueName: \"kubernetes.io/projected/5e825ce1-2ad9-4309-a65f-620303b66e0e-kube-api-access-rbs9z\") pod \"test-curl-7572-139675858425648\" (UID: \"5e825ce1-2ad9-4309-a65f-620303b66e0e\") " pod="opendatahub/test-curl-7572-139675858425648"
Jul 28 21:00:46.911483 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:46.911462    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858425648"
Jul 28 21:00:47.043878 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:47.043854    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7572-139675858425648"]
Jul 28 21:00:47.045982 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:00:47.045956    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e825ce1_2ad9_4309_a65f_620303b66e0e.slice/crio-c7e19101a0c16860d20aef2d7b97cfc1c78d876e80559f5496cca8565a21b1dd WatchSource:0}: Error finding container c7e19101a0c16860d20aef2d7b97cfc1c78d876e80559f5496cca8565a21b1dd: Status 404 returned error can't find the container with id c7e19101a0c16860d20aef2d7b97cfc1c78d876e80559f5496cca8565a21b1dd
Jul 28 21:00:47.247237 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:47.247158    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858378416"
Jul 28 21:00:47.247237 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:47.247196    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b00dc143dd02c53ba61db257c870ffc3d17c69767d6e19f3ccd04bc47e5f374"
Jul 28 21:00:47.248643 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:47.248609    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7572-139675858425648" event={"ID":"5e825ce1-2ad9-4309-a65f-620303b66e0e","Type":"ContainerStarted","Data":"c7e19101a0c16860d20aef2d7b97cfc1c78d876e80559f5496cca8565a21b1dd"}
Jul 28 21:00:47.258281 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:00:47.258248    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7572-139675858378416\" is forbidden: User \"system:node:ip-10-0-133-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-72.ec2.internal' and this object" podUID="39efa226-17d2-46d4-b3b2-be61ea2150bf" pod="opendatahub/test-curl-7572-139675858378416"
Jul 28 21:00:47.340985 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:47.340957    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39efa226-17d2-46d4-b3b2-be61ea2150bf" path="/var/lib/kubelet/pods/39efa226-17d2-46d4-b3b2-be61ea2150bf/volumes"
Jul 28 21:00:48.255003 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:48.254964    2558 generic.go:356] "Generic (PLEG): container finished" podID="5e825ce1-2ad9-4309-a65f-620303b66e0e" containerID="0af4715a00fe0b842f3426b283447989a9e68b44b16b3bd16b344f3cf0f2abee" exitCode=0
Jul 28 21:00:48.255454 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:48.255018    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7572-139675858425648" event={"ID":"5e825ce1-2ad9-4309-a65f-620303b66e0e","Type":"ContainerDied","Data":"0af4715a00fe0b842f3426b283447989a9e68b44b16b3bd16b344f3cf0f2abee"}
Jul 28 21:00:49.398091 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.398065    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858425648"
Jul 28 21:00:49.419028 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.419004    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7572-139675858425648_5e825ce1-2ad9-4309-a65f-620303b66e0e/test-curl-7572-139675858425648/0.log"
Jul 28 21:00:49.444603 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.444575    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7572-139675858425648"]
Jul 28 21:00:49.448330 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.448306    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7572-139675858425648"]
Jul 28 21:00:49.495451 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.495431    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbs9z\" (UniqueName: \"kubernetes.io/projected/5e825ce1-2ad9-4309-a65f-620303b66e0e-kube-api-access-rbs9z\") pod \"5e825ce1-2ad9-4309-a65f-620303b66e0e\" (UID: \"5e825ce1-2ad9-4309-a65f-620303b66e0e\") "
Jul 28 21:00:49.497614 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.497591    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e825ce1-2ad9-4309-a65f-620303b66e0e-kube-api-access-rbs9z" (OuterVolumeSpecName: "kube-api-access-rbs9z") pod "5e825ce1-2ad9-4309-a65f-620303b66e0e" (UID: "5e825ce1-2ad9-4309-a65f-620303b66e0e"). InnerVolumeSpecName "kube-api-access-rbs9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:00:49.597032 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.596966    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbs9z\" (UniqueName: \"kubernetes.io/projected/5e825ce1-2ad9-4309-a65f-620303b66e0e-kube-api-access-rbs9z\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:00:49.615992 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.615970    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7572-139675858462736"]
Jul 28 21:00:49.616331 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.616319    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e825ce1-2ad9-4309-a65f-620303b66e0e" containerName="test-curl-7572-139675858425648"
Jul 28 21:00:49.616374 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.616332    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e825ce1-2ad9-4309-a65f-620303b66e0e" containerName="test-curl-7572-139675858425648"
Jul 28 21:00:49.616414 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.616405    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e825ce1-2ad9-4309-a65f-620303b66e0e" containerName="test-curl-7572-139675858425648"
Jul 28 21:00:49.619372 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.619357    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:49.623826 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.623804    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7572-139675858462736"]
Jul 28 21:00:49.798044 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.798013    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r86bf\" (UniqueName: \"kubernetes.io/projected/e5ec4b1d-7675-473f-9640-9c11c6856cf4-kube-api-access-r86bf\") pod \"test-curl-7572-139675858462736\" (UID: \"e5ec4b1d-7675-473f-9640-9c11c6856cf4\") " pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:49.898774 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.898694    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r86bf\" (UniqueName: \"kubernetes.io/projected/e5ec4b1d-7675-473f-9640-9c11c6856cf4-kube-api-access-r86bf\") pod \"test-curl-7572-139675858462736\" (UID: \"e5ec4b1d-7675-473f-9640-9c11c6856cf4\") " pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:49.906987 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.906963    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r86bf\" (UniqueName: \"kubernetes.io/projected/e5ec4b1d-7675-473f-9640-9c11c6856cf4-kube-api-access-r86bf\") pod \"test-curl-7572-139675858462736\" (UID: \"e5ec4b1d-7675-473f-9640-9c11c6856cf4\") " pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:49.930326 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:49.930305    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:50.053969 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:50.053936    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7572-139675858462736"]
Jul 28 21:00:50.266116 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:50.266091    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858425648"
Jul 28 21:00:50.266116 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:50.266115    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7e19101a0c16860d20aef2d7b97cfc1c78d876e80559f5496cca8565a21b1dd"
Jul 28 21:00:50.267346 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:50.267317    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7572-139675858462736" event={"ID":"e5ec4b1d-7675-473f-9640-9c11c6856cf4","Type":"ContainerStarted","Data":"ebe17cde8875aa5f460f6508104ebdeb28970664a88354e07a762f00a66d7ed3"}
Jul 28 21:00:50.276460 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:00:50.276434    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7572-139675858425648\" is forbidden: User \"system:node:ip-10-0-133-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-72.ec2.internal' and this object" podUID="5e825ce1-2ad9-4309-a65f-620303b66e0e" pod="opendatahub/test-curl-7572-139675858425648"
Jul 28 21:00:51.273491 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:51.273451    2558 generic.go:356] "Generic (PLEG): container finished" podID="e5ec4b1d-7675-473f-9640-9c11c6856cf4" containerID="ed7ab9731bf51e0cc0c1a6e5c162d676881397268a9125e077fe5ce248be660f" exitCode=0
Jul 28 21:00:51.273884 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:51.273519    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7572-139675858462736" event={"ID":"e5ec4b1d-7675-473f-9640-9c11c6856cf4","Type":"ContainerDied","Data":"ed7ab9731bf51e0cc0c1a6e5c162d676881397268a9125e077fe5ce248be660f"}
Jul 28 21:00:51.293285 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:00:51.293250    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7572-139675858425648\" is forbidden: User \"system:node:ip-10-0-133-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-72.ec2.internal' and this object" podUID="5e825ce1-2ad9-4309-a65f-620303b66e0e" pod="opendatahub/test-curl-7572-139675858425648"
Jul 28 21:00:51.340258 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:51.340216    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e825ce1-2ad9-4309-a65f-620303b66e0e" path="/var/lib/kubelet/pods/5e825ce1-2ad9-4309-a65f-620303b66e0e/volumes"
Jul 28 21:00:52.408269 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.408243    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:52.427801 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.427783    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7572-139675858462736_e5ec4b1d-7675-473f-9640-9c11c6856cf4/test-curl-7572-139675858462736/0.log"
Jul 28 21:00:52.451271 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.451246    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7572-139675858462736"]
Jul 28 21:00:52.454796 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.454774    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7572-139675858462736"]
Jul 28 21:00:52.519333 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.519312    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r86bf\" (UniqueName: \"kubernetes.io/projected/e5ec4b1d-7675-473f-9640-9c11c6856cf4-kube-api-access-r86bf\") pod \"e5ec4b1d-7675-473f-9640-9c11c6856cf4\" (UID: \"e5ec4b1d-7675-473f-9640-9c11c6856cf4\") "
Jul 28 21:00:52.521388 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.521367    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5ec4b1d-7675-473f-9640-9c11c6856cf4-kube-api-access-r86bf" (OuterVolumeSpecName: "kube-api-access-r86bf") pod "e5ec4b1d-7675-473f-9640-9c11c6856cf4" (UID: "e5ec4b1d-7675-473f-9640-9c11c6856cf4"). InnerVolumeSpecName "kube-api-access-r86bf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:00:52.582132 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.582080    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7572-139675858462736"]
Jul 28 21:00:52.582451 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.582439    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ec4b1d-7675-473f-9640-9c11c6856cf4" containerName="test-curl-7572-139675858462736"
Jul 28 21:00:52.582492 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.582452    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ec4b1d-7675-473f-9640-9c11c6856cf4" containerName="test-curl-7572-139675858462736"
Jul 28 21:00:52.582527 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.582518    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ec4b1d-7675-473f-9640-9c11c6856cf4" containerName="test-curl-7572-139675858462736"
Jul 28 21:00:52.585435 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.585420    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:52.591718 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.591698    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7572-139675858462736"]
Jul 28 21:00:52.620624 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.620599    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r86bf\" (UniqueName: \"kubernetes.io/projected/e5ec4b1d-7675-473f-9640-9c11c6856cf4-kube-api-access-r86bf\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:00:52.721245 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.721222    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwlp2\" (UniqueName: \"kubernetes.io/projected/deea1e99-acb4-41ef-bbb0-9d326900a16b-kube-api-access-rwlp2\") pod \"test-curl-7572-139675858462736\" (UID: \"deea1e99-acb4-41ef-bbb0-9d326900a16b\") " pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:52.822749 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.822719    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwlp2\" (UniqueName: \"kubernetes.io/projected/deea1e99-acb4-41ef-bbb0-9d326900a16b-kube-api-access-rwlp2\") pod \"test-curl-7572-139675858462736\" (UID: \"deea1e99-acb4-41ef-bbb0-9d326900a16b\") " pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:52.830152 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.830129    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwlp2\" (UniqueName: \"kubernetes.io/projected/deea1e99-acb4-41ef-bbb0-9d326900a16b-kube-api-access-rwlp2\") pod \"test-curl-7572-139675858462736\" (UID: \"deea1e99-acb4-41ef-bbb0-9d326900a16b\") " pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:52.895711 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:52.895646    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:53.021768 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:53.021739    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7572-139675858462736"]
Jul 28 21:00:53.023859 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:00:53.023831    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddeea1e99_acb4_41ef_bbb0_9d326900a16b.slice/crio-16bd1b701ce27c587f6129568715bf32e1d4125711a249fc4d193af0b6a6fe11 WatchSource:0}: Error finding container 16bd1b701ce27c587f6129568715bf32e1d4125711a249fc4d193af0b6a6fe11: Status 404 returned error can't find the container with id 16bd1b701ce27c587f6129568715bf32e1d4125711a249fc4d193af0b6a6fe11
Jul 28 21:00:53.284539 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:53.284505    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7572-139675858462736" event={"ID":"deea1e99-acb4-41ef-bbb0-9d326900a16b","Type":"ContainerStarted","Data":"16bd1b701ce27c587f6129568715bf32e1d4125711a249fc4d193af0b6a6fe11"}
Jul 28 21:00:53.285736 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:53.285722    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebe17cde8875aa5f460f6508104ebdeb28970664a88354e07a762f00a66d7ed3"
Jul 28 21:00:53.285736 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:53.285730    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:53.297244 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:53.297217    2558 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7572-139675858462736" oldPodUID="e5ec4b1d-7675-473f-9640-9c11c6856cf4" podUID="deea1e99-acb4-41ef-bbb0-9d326900a16b"
Jul 28 21:00:53.339978 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:53.339936    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5ec4b1d-7675-473f-9640-9c11c6856cf4" path="/var/lib/kubelet/pods/e5ec4b1d-7675-473f-9640-9c11c6856cf4/volumes"
Jul 28 21:00:54.292435 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.292397    2558 generic.go:356] "Generic (PLEG): container finished" podID="deea1e99-acb4-41ef-bbb0-9d326900a16b" containerID="b4780d69f342f149640cd0c11fb124ba6c722d00c7b34748a9b41e4ff3f36fdd" exitCode=0
Jul 28 21:00:54.292810 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.292460    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7572-139675858462736" event={"ID":"deea1e99-acb4-41ef-bbb0-9d326900a16b","Type":"ContainerDied","Data":"b4780d69f342f149640cd0c11fb124ba6c722d00c7b34748a9b41e4ff3f36fdd"}
Jul 28 21:00:54.600114 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.600024    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd" podUID="fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac" containerName="main" containerID="cri-o://91dde08e4c806922fcae5b3f713051856c350165bd7d2a7d7808cc26e209ba4c" gracePeriod=45
Jul 28 21:00:54.853790 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.853732    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 21:00:54.942998 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.942969    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-dshm\") pod \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") "
Jul 28 21:00:54.943119 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.943009    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-home\") pod \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") "
Jul 28 21:00:54.943119 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.943031    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-tmp-dir\") pod \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") "
Jul 28 21:00:54.943201 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.943126    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-tls-certs\") pod \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") "
Jul 28 21:00:54.943201 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.943161    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-model-cache\") pod \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") "
Jul 28 21:00:54.943201 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.943191    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7pl9\" (UniqueName: \"kubernetes.io/projected/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-kube-api-access-k7pl9\") pod \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\" (UID: \"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac\") "
Jul 28 21:00:54.943414 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.943264    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-home" (OuterVolumeSpecName: "home") pod "fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac" (UID: "fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:00:54.943414 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.943299    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac" (UID: "fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:00:54.943518 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.943463    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac" (UID: "fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:00:54.943624 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.943607    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-home\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:00:54.943688 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.943630    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-tmp-dir\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:00:54.943688 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.943644    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-model-cache\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:00:54.945234 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.945207    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-kube-api-access-k7pl9" (OuterVolumeSpecName: "kube-api-access-k7pl9") pod "fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac" (UID: "fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac"). InnerVolumeSpecName "kube-api-access-k7pl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:00:54.945331 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.945247    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-dshm" (OuterVolumeSpecName: "dshm") pod "fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac" (UID: "fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:00:54.945331 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:54.945264    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac" (UID: "fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:00:55.044977 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.044941    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-tls-certs\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:00:55.044977 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.044977    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7pl9\" (UniqueName: \"kubernetes.io/projected/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-kube-api-access-k7pl9\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:00:55.044977 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.044986    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac-dshm\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:00:55.298611 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.298581    2558 generic.go:356] "Generic (PLEG): container finished" podID="fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac" containerID="91dde08e4c806922fcae5b3f713051856c350165bd7d2a7d7808cc26e209ba4c" exitCode=0
Jul 28 21:00:55.298985 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.298616    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"
Jul 28 21:00:55.298985 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.298663    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd" event={"ID":"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac","Type":"ContainerDied","Data":"91dde08e4c806922fcae5b3f713051856c350165bd7d2a7d7808cc26e209ba4c"}
Jul 28 21:00:55.298985 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.298702    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd" event={"ID":"fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac","Type":"ContainerDied","Data":"1ae4e068203319bcaa6def5c4627752333d21b5db441893b7beecbdbc02e059f"}
Jul 28 21:00:55.298985 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.298722    2558 scope.go:117] "RemoveContainer" containerID="91dde08e4c806922fcae5b3f713051856c350165bd7d2a7d7808cc26e209ba4c"
Jul 28 21:00:55.308465 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.308449    2558 scope.go:117] "RemoveContainer" containerID="91dde08e4c806922fcae5b3f713051856c350165bd7d2a7d7808cc26e209ba4c"
Jul 28 21:00:55.308722 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:00:55.308697    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91dde08e4c806922fcae5b3f713051856c350165bd7d2a7d7808cc26e209ba4c\": container with ID starting with 91dde08e4c806922fcae5b3f713051856c350165bd7d2a7d7808cc26e209ba4c not found: ID does not exist" containerID="91dde08e4c806922fcae5b3f713051856c350165bd7d2a7d7808cc26e209ba4c"
Jul 28 21:00:55.308833 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.308727    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91dde08e4c806922fcae5b3f713051856c350165bd7d2a7d7808cc26e209ba4c"} err="failed to get container status \"91dde08e4c806922fcae5b3f713051856c350165bd7d2a7d7808cc26e209ba4c\": rpc error: code = NotFound desc = could not find container \"91dde08e4c806922fcae5b3f713051856c350165bd7d2a7d7808cc26e209ba4c\": container with ID starting with 91dde08e4c806922fcae5b3f713051856c350165bd7d2a7d7808cc26e209ba4c not found: ID does not exist"
Jul 28 21:00:55.319975 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.319953    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"]
Jul 28 21:00:55.323960 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.323940    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-cc370c/rate-test-model-cc370c-kserve-6b78847f95-p8vsd"]
Jul 28 21:00:55.340272 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.340248    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac" path="/var/lib/kubelet/pods/fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac/volumes"
Jul 28 21:00:55.423503 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.423480    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:55.443862 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.443844    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7572-139675858462736_deea1e99-acb4-41ef-bbb0-9d326900a16b/test-curl-7572-139675858462736/0.log"
Jul 28 21:00:55.448123 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.448105    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwlp2\" (UniqueName: \"kubernetes.io/projected/deea1e99-acb4-41ef-bbb0-9d326900a16b-kube-api-access-rwlp2\") pod \"deea1e99-acb4-41ef-bbb0-9d326900a16b\" (UID: \"deea1e99-acb4-41ef-bbb0-9d326900a16b\") "
Jul 28 21:00:55.450040 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.450018    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deea1e99-acb4-41ef-bbb0-9d326900a16b-kube-api-access-rwlp2" (OuterVolumeSpecName: "kube-api-access-rwlp2") pod "deea1e99-acb4-41ef-bbb0-9d326900a16b" (UID: "deea1e99-acb4-41ef-bbb0-9d326900a16b"). InnerVolumeSpecName "kube-api-access-rwlp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:00:55.474419 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.474398    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7572-139675858462736"]
Jul 28 21:00:55.479077 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.479059    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7572-139675858462736"]
Jul 28 21:00:55.548939 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.548892    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwlp2\" (UniqueName: \"kubernetes.io/projected/deea1e99-acb4-41ef-bbb0-9d326900a16b-kube-api-access-rwlp2\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:00:55.607966 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.607944    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7572-139675858463744"]
Jul 28 21:00:55.608404 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.608389    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac" containerName="main"
Jul 28 21:00:55.608449 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.608406    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac" containerName="main"
Jul 28 21:00:55.608449 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.608428    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="deea1e99-acb4-41ef-bbb0-9d326900a16b" containerName="test-curl-7572-139675858462736"
Jul 28 21:00:55.608449 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.608434    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="deea1e99-acb4-41ef-bbb0-9d326900a16b" containerName="test-curl-7572-139675858462736"
Jul 28 21:00:55.608572 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.608496    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="deea1e99-acb4-41ef-bbb0-9d326900a16b" containerName="test-curl-7572-139675858462736"
Jul 28 21:00:55.608572 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.608503    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd238da7-bf12-40eb-aeb1-fa8b8e80a9ac" containerName="main"
Jul 28 21:00:55.611525 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.611510    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:00:55.618461 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.618439    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7572-139675858463744"]
Jul 28 21:00:55.649691 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.649665    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrcz8\" (UniqueName: \"kubernetes.io/projected/b0e1d766-499f-480d-9acb-e834081e14af-kube-api-access-xrcz8\") pod \"test-curl-7572-139675858463744\" (UID: \"b0e1d766-499f-480d-9acb-e834081e14af\") " pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:00:55.750704 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.750678    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrcz8\" (UniqueName: \"kubernetes.io/projected/b0e1d766-499f-480d-9acb-e834081e14af-kube-api-access-xrcz8\") pod \"test-curl-7572-139675858463744\" (UID: \"b0e1d766-499f-480d-9acb-e834081e14af\") " pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:00:55.758854 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.758831    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrcz8\" (UniqueName: \"kubernetes.io/projected/b0e1d766-499f-480d-9acb-e834081e14af-kube-api-access-xrcz8\") pod \"test-curl-7572-139675858463744\" (UID: \"b0e1d766-499f-480d-9acb-e834081e14af\") " pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:00:55.925889 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:55.925830    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:00:56.266163 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:56.266139    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7572-139675858463744"]
Jul 28 21:00:56.267979 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:00:56.267949    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0e1d766_499f_480d_9acb_e834081e14af.slice/crio-c1bb08197403dc969ccd84f1abb8e146a95f9716fb289959edccf74e4715352b WatchSource:0}: Error finding container c1bb08197403dc969ccd84f1abb8e146a95f9716fb289959edccf74e4715352b: Status 404 returned error can't find the container with id c1bb08197403dc969ccd84f1abb8e146a95f9716fb289959edccf74e4715352b
Jul 28 21:00:56.306356 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:56.306329    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7572-139675858463744" event={"ID":"b0e1d766-499f-480d-9acb-e834081e14af","Type":"ContainerStarted","Data":"c1bb08197403dc969ccd84f1abb8e146a95f9716fb289959edccf74e4715352b"}
Jul 28 21:00:56.307601 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:56.307579    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16bd1b701ce27c587f6129568715bf32e1d4125711a249fc4d193af0b6a6fe11"
Jul 28 21:00:56.307601 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:56.307588    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:56.317878 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:00:56.317852    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7572-139675858462736\" is forbidden: User \"system:node:ip-10-0-133-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-72.ec2.internal' and this object" podUID="deea1e99-acb4-41ef-bbb0-9d326900a16b" pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:57.314373 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:57.314342    2558 generic.go:356] "Generic (PLEG): container finished" podID="b0e1d766-499f-480d-9acb-e834081e14af" containerID="a99657f5641fdc48b8e2f39010aa53487ef8d2a5e6dbadbabe4b5fe78ae2ebed" exitCode=0
Jul 28 21:00:57.314806 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:57.314428    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7572-139675858463744" event={"ID":"b0e1d766-499f-480d-9acb-e834081e14af","Type":"ContainerDied","Data":"a99657f5641fdc48b8e2f39010aa53487ef8d2a5e6dbadbabe4b5fe78ae2ebed"}
Jul 28 21:00:57.329885 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:00:57.329851    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7572-139675858462736\" is forbidden: User \"system:node:ip-10-0-133-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-72.ec2.internal' and this object" podUID="deea1e99-acb4-41ef-bbb0-9d326900a16b" pod="opendatahub/test-curl-7572-139675858462736"
Jul 28 21:00:57.339655 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:57.339632    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="deea1e99-acb4-41ef-bbb0-9d326900a16b" path="/var/lib/kubelet/pods/deea1e99-acb4-41ef-bbb0-9d326900a16b/volumes"
Jul 28 21:00:58.458650 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.458625    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:00:58.471093 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.471071    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrcz8\" (UniqueName: \"kubernetes.io/projected/b0e1d766-499f-480d-9acb-e834081e14af-kube-api-access-xrcz8\") pod \"b0e1d766-499f-480d-9acb-e834081e14af\" (UID: \"b0e1d766-499f-480d-9acb-e834081e14af\") "
Jul 28 21:00:58.472870 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.472840    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0e1d766-499f-480d-9acb-e834081e14af-kube-api-access-xrcz8" (OuterVolumeSpecName: "kube-api-access-xrcz8") pod "b0e1d766-499f-480d-9acb-e834081e14af" (UID: "b0e1d766-499f-480d-9acb-e834081e14af"). InnerVolumeSpecName "kube-api-access-xrcz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:00:58.480667 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.480649    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7572-139675858463744_b0e1d766-499f-480d-9acb-e834081e14af/test-curl-7572-139675858463744/0.log"
Jul 28 21:00:58.507540 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.507515    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7572-139675858463744"]
Jul 28 21:00:58.512675 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.512656    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7572-139675858463744"]
Jul 28 21:00:58.571906 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.571852    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrcz8\" (UniqueName: \"kubernetes.io/projected/b0e1d766-499f-480d-9acb-e834081e14af-kube-api-access-xrcz8\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:00:58.637452 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.637429    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7572-139675858463744"]
Jul 28 21:00:58.637830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.637817    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0e1d766-499f-480d-9acb-e834081e14af" containerName="test-curl-7572-139675858463744"
Jul 28 21:00:58.637888 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.637832    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0e1d766-499f-480d-9acb-e834081e14af" containerName="test-curl-7572-139675858463744"
Jul 28 21:00:58.637923 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.637906    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0e1d766-499f-480d-9acb-e834081e14af" containerName="test-curl-7572-139675858463744"
Jul 28 21:00:58.640828 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.640811    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:00:58.650032 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.650007    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7572-139675858463744"]
Jul 28 21:00:58.672548 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.672523    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4rkh\" (UniqueName: \"kubernetes.io/projected/9fe7aee0-6e22-49b2-b928-e0f256c86328-kube-api-access-c4rkh\") pod \"test-curl-7572-139675858463744\" (UID: \"9fe7aee0-6e22-49b2-b928-e0f256c86328\") " pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:00:58.773491 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.773463    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4rkh\" (UniqueName: \"kubernetes.io/projected/9fe7aee0-6e22-49b2-b928-e0f256c86328-kube-api-access-c4rkh\") pod \"test-curl-7572-139675858463744\" (UID: \"9fe7aee0-6e22-49b2-b928-e0f256c86328\") " pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:00:58.781708 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.781680    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4rkh\" (UniqueName: \"kubernetes.io/projected/9fe7aee0-6e22-49b2-b928-e0f256c86328-kube-api-access-c4rkh\") pod \"test-curl-7572-139675858463744\" (UID: \"9fe7aee0-6e22-49b2-b928-e0f256c86328\") " pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:00:58.950941 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:58.950872    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:00:59.108098 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:59.108067    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7572-139675858463744"]
Jul 28 21:00:59.111098 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:00:59.111075    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fe7aee0_6e22_49b2_b928_e0f256c86328.slice/crio-f5e2c3ef063237c5a62869968388bb85c31e8c16db207aa91c2a9bd42388d4d4 WatchSource:0}: Error finding container f5e2c3ef063237c5a62869968388bb85c31e8c16db207aa91c2a9bd42388d4d4: Status 404 returned error can't find the container with id f5e2c3ef063237c5a62869968388bb85c31e8c16db207aa91c2a9bd42388d4d4
Jul 28 21:00:59.326375 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:59.326350    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:00:59.326499 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:59.326350    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1bb08197403dc969ccd84f1abb8e146a95f9716fb289959edccf74e4715352b"
Jul 28 21:00:59.327596 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:59.327573    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7572-139675858463744" event={"ID":"9fe7aee0-6e22-49b2-b928-e0f256c86328","Type":"ContainerStarted","Data":"f5e2c3ef063237c5a62869968388bb85c31e8c16db207aa91c2a9bd42388d4d4"}
Jul 28 21:00:59.342026 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:59.337915    2558 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7572-139675858463744" oldPodUID="b0e1d766-499f-480d-9acb-e834081e14af" podUID="9fe7aee0-6e22-49b2-b928-e0f256c86328"
Jul 28 21:00:59.345997 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:00:59.345976    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0e1d766-499f-480d-9acb-e834081e14af" path="/var/lib/kubelet/pods/b0e1d766-499f-480d-9acb-e834081e14af/volumes"
Jul 28 21:01:00.334045 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:00.334012    2558 generic.go:356] "Generic (PLEG): container finished" podID="9fe7aee0-6e22-49b2-b928-e0f256c86328" containerID="9b84dc190171dc8118e88aca8b2247ae6a2984550242eecfca65e35ba6735b0a" exitCode=0
Jul 28 21:01:00.334430 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:00.334082    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7572-139675858463744" event={"ID":"9fe7aee0-6e22-49b2-b928-e0f256c86328","Type":"ContainerDied","Data":"9b84dc190171dc8118e88aca8b2247ae6a2984550242eecfca65e35ba6735b0a"}
Jul 28 21:01:01.474094 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:01.474067    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:01:01.494034 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:01.494008    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7572-139675858463744_9fe7aee0-6e22-49b2-b928-e0f256c86328/test-curl-7572-139675858463744/0.log"
Jul 28 21:01:01.496773 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:01.496746    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4rkh\" (UniqueName: \"kubernetes.io/projected/9fe7aee0-6e22-49b2-b928-e0f256c86328-kube-api-access-c4rkh\") pod \"9fe7aee0-6e22-49b2-b928-e0f256c86328\" (UID: \"9fe7aee0-6e22-49b2-b928-e0f256c86328\") "
Jul 28 21:01:01.498678 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:01.498657    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe7aee0-6e22-49b2-b928-e0f256c86328-kube-api-access-c4rkh" (OuterVolumeSpecName: "kube-api-access-c4rkh") pod "9fe7aee0-6e22-49b2-b928-e0f256c86328" (UID: "9fe7aee0-6e22-49b2-b928-e0f256c86328"). InnerVolumeSpecName "kube-api-access-c4rkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:01:01.521443 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:01.521416    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7572-139675858463744"]
Jul 28 21:01:01.525247 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:01.525227    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7572-139675858463744"]
Jul 28 21:01:01.597651 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:01.597594    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4rkh\" (UniqueName: \"kubernetes.io/projected/9fe7aee0-6e22-49b2-b928-e0f256c86328-kube-api-access-c4rkh\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:01:02.346547 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:02.346516    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:01:02.346547 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:02.346548    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5e2c3ef063237c5a62869968388bb85c31e8c16db207aa91c2a9bd42388d4d4"
Jul 28 21:01:02.369444 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:01:02.369409    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7572-139675858463744\" is forbidden: User \"system:node:ip-10-0-133-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-72.ec2.internal' and this object" podUID="9fe7aee0-6e22-49b2-b928-e0f256c86328" pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:01:03.339729 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:01:03.339687    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7572-139675858463744\" is forbidden: User \"system:node:ip-10-0-133-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-72.ec2.internal' and this object" podUID="9fe7aee0-6e22-49b2-b928-e0f256c86328" pod="opendatahub/test-curl-7572-139675858463744"
Jul 28 21:01:03.340923 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:03.340902    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fe7aee0-6e22-49b2-b928-e0f256c86328" path="/var/lib/kubelet/pods/9fe7aee0-6e22-49b2-b928-e0f256c86328/volumes"
Jul 28 21:01:14.834268 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:14.834186    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:01:31.679420 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.679386    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"]
Jul 28 21:01:31.680020 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.679998    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fe7aee0-6e22-49b2-b928-e0f256c86328" containerName="test-curl-7572-139675858463744"
Jul 28 21:01:31.680106 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.680024    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe7aee0-6e22-49b2-b928-e0f256c86328" containerName="test-curl-7572-139675858463744"
Jul 28 21:01:31.680188 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.680171    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fe7aee0-6e22-49b2-b928-e0f256c86328" containerName="test-curl-7572-139675858463744"
Jul 28 21:01:31.683603 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.683583    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.686285 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.686265    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 21:01:31.686376 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.686285    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-542599-openshift-default-dockercfg-q7q9t\""
Jul 28 21:01:31.699003 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.698977    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"]
Jul 28 21:01:31.846538 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.846512    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-workload-socket\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.846684 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.846542    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6tgx\" (UniqueName: \"kubernetes.io/projected/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-kube-api-access-r6tgx\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.846684 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.846590    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-podinfo\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.846684 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.846611    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-workload-certs\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.846684 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.846628    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-envoy\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.846823 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.846681    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istiod-ca-cert\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.846823 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.846714    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-credential-socket\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.846823 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.846765    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-token\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.846823 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.846785    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-data\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948055 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.947984    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-podinfo\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948055 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.948023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-workload-certs\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948055 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.948050    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-envoy\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948284 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.948095    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istiod-ca-cert\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948284 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.948122    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-credential-socket\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948284 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.948161    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-token\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948284 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.948188    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-data\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948284 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.948263    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-workload-socket\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948546 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.948294    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6tgx\" (UniqueName: \"kubernetes.io/projected/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-kube-api-access-r6tgx\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948546 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.948429    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-workload-certs\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948546 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.948487    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-credential-socket\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948774 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.948752    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-workload-socket\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948819 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.948794    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-data\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.948882 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.948865    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istiod-ca-cert\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.950417 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.950397    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-envoy\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.950664 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.950647    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-podinfo\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.956142 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.956119    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6tgx\" (UniqueName: \"kubernetes.io/projected/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-kube-api-access-r6tgx\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.956219 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.956124    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-token\") pod \"e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") " pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:31.994376 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:31.994354    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:32.129977 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:32.129951    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"]
Jul 28 21:01:32.131473 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:01:32.131447    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod009b6a70_90cb_4c9e_8903_a0ff2d2b654f.slice/crio-714cbdea00231d2f1d619f9fdfb0657e4efbe59a55a377472979f1df4e68fb61 WatchSource:0}: Error finding container 714cbdea00231d2f1d619f9fdfb0657e4efbe59a55a377472979f1df4e68fb61: Status 404 returned error can't find the container with id 714cbdea00231d2f1d619f9fdfb0657e4efbe59a55a377472979f1df4e68fb61
Jul 28 21:01:32.507595 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:32.507542    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5" event={"ID":"009b6a70-90cb-4c9e-8903-a0ff2d2b654f","Type":"ContainerStarted","Data":"714cbdea00231d2f1d619f9fdfb0657e4efbe59a55a377472979f1df4e68fb61"}
Jul 28 21:01:35.068009 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:35.067970    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 21:01:35.068290 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:35.068048    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 21:01:35.068290 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:35.068074    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 21:01:35.527941 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:35.527900    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5" event={"ID":"009b6a70-90cb-4c9e-8903-a0ff2d2b654f","Type":"ContainerStarted","Data":"0ae68ed746f4af8fb97b4639573b366cc81049a8074257fbaacc7db776844ab2"}
Jul 28 21:01:35.550005 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:35.549950    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5" podStartSLOduration=1.615828813 podStartE2EDuration="4.549930276s" totalImagesPullingTime="2.934101463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:01:35.548311293 +0000 UTC m=+3172.790404517" watchObservedRunningTime="2026-07-28 21:01:35.549930276 +0000 UTC m=+3172.792023500"
Jul 28 21:01:35.994752 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:35.994660    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:37.000728 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:37.000693    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:37.538230 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:37.538202    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:37.539192 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:37.539170    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:44.098337 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:44.098304    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:01:52.539934 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:52.539898    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 28 21:01:52.540328 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:52.539954    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5" podUID="009b6a70-90cb-4c9e-8903-a0ff2d2b654f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 21:01:52.621124 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:52.621096    2558 generic.go:356] "Generic (PLEG): container finished" podID="009b6a70-90cb-4c9e-8903-a0ff2d2b654f" containerID="0ae68ed746f4af8fb97b4639573b366cc81049a8074257fbaacc7db776844ab2" exitCode=0
Jul 28 21:01:52.621271 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:52.621177    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5" event={"ID":"009b6a70-90cb-4c9e-8903-a0ff2d2b654f","Type":"ContainerDied","Data":"0ae68ed746f4af8fb97b4639573b366cc81049a8074257fbaacc7db776844ab2"}
Jul 28 21:01:52.621640 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:52.621625    2558 scope.go:117] "RemoveContainer" containerID="0ae68ed746f4af8fb97b4639573b366cc81049a8074257fbaacc7db776844ab2"
Jul 28 21:01:52.622770 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:52.622744    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 21:01:52.622837 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:52.622812    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 21:01:52.622881 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:52.622842    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 21:01:53.632789 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:53.632745    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5" event={"ID":"009b6a70-90cb-4c9e-8903-a0ff2d2b654f","Type":"ContainerStarted","Data":"b375eac2c2453a629e59d0525d76467badeea060d5a959689fd93fb4c33a3e01"}
Jul 28 21:01:53.994826 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:53.994726    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:53.994995 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:53.994936    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 28 21:01:53.994995 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:53.994983    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5" podUID="009b6a70-90cb-4c9e-8903-a0ff2d2b654f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 21:01:54.994883 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:54.994843    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 28 21:01:54.995274 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:54.994906    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5" podUID="009b6a70-90cb-4c9e-8903-a0ff2d2b654f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 21:01:55.995441 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:55.995409    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 28 21:01:55.995852 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:55.995460    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5" podUID="009b6a70-90cb-4c9e-8903-a0ff2d2b654f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 21:01:56.994879 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:56.994839    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 28 21:01:56.995059 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:56.994898    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5" podUID="009b6a70-90cb-4c9e-8903-a0ff2d2b654f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 28 21:01:57.999339 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:57.999306    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:57.999724 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:57.999569    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:01:58.000310 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:01:58.000288    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:02:24.789174 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:24.789136    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"]
Jul 28 21:02:24.792747 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:24.792727    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:24.797226 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:24.797199    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-ce337a\"/\"kube-root-ca.crt\""
Jul 28 21:02:24.797226 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:24.797217    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-ce337a\"/\"openshift-service-ca.crt\""
Jul 28 21:02:24.797420 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:24.797225    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-ce337a\"/\"ipp-route-ce337a-kserve-self-signed-certs\""
Jul 28 21:02:24.797420 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:24.797339    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-ce337a\"/\"default-dockercfg-wcdfj\""
Jul 28 21:02:24.803340 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:24.803319    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"]
Jul 28 21:02:24.921178 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:24.921142    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjbjq\" (UniqueName: \"kubernetes.io/projected/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-kube-api-access-qjbjq\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:24.921351 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:24.921215    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-tmp-dir\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:24.921351 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:24.921239    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-tls-certs\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:24.921351 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:24.921268    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-home\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:24.921470 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:24.921345    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-model-cache\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:24.921470 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:24.921420    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-dshm\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.022716 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.022681    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-home\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.022716 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.022720    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-model-cache\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.022930 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.022769    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-dshm\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.022930 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.022802    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjbjq\" (UniqueName: \"kubernetes.io/projected/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-kube-api-access-qjbjq\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.023042 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.022970    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-tmp-dir\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.023042 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.023026    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-tls-certs\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.023157 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.023135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-home\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.023216 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.023153    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-model-cache\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.023308 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.023290    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-tmp-dir\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.025029 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.024995    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-dshm\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.025426 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.025405    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-tls-certs\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.030821 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.030799    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjbjq\" (UniqueName: \"kubernetes.io/projected/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-kube-api-access-qjbjq\") pod \"ipp-route-ce337a-kserve-5d84c678c5-7wwj8\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") " pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.102844 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.102777    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:25.232875 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.232847    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"]
Jul 28 21:02:25.235074 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:02:25.235049    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb1d11d7_4c1d_4b88_9516_8ecd51d72f00.slice/crio-5af60a3222ed4ba4feb10d615d9c8b0c21330a2ec383b6e2616d76b6a61d4bd3 WatchSource:0}: Error finding container 5af60a3222ed4ba4feb10d615d9c8b0c21330a2ec383b6e2616d76b6a61d4bd3: Status 404 returned error can't find the container with id 5af60a3222ed4ba4feb10d615d9c8b0c21330a2ec383b6e2616d76b6a61d4bd3
Jul 28 21:02:25.808184 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.808142    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8" event={"ID":"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00","Type":"ContainerStarted","Data":"5122ffb90f3881164dec4a4078607a81aec719749885672a42fd98ce3045c822"}
Jul 28 21:02:25.808573 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.808193    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8" event={"ID":"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00","Type":"ContainerStarted","Data":"5af60a3222ed4ba4feb10d615d9c8b0c21330a2ec383b6e2616d76b6a61d4bd3"}
Jul 28 21:02:25.826566 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:25.826517    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8" podStartSLOduration=1.826501634 podStartE2EDuration="1.826501634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:02:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:02:25.824632362 +0000 UTC m=+3223.066725584" watchObservedRunningTime="2026-07-28 21:02:25.826501634 +0000 UTC m=+3223.068594856"
Jul 28 21:02:28.497933 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:28.497900    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:02:35.103296 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:35.103220    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:35.103296 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:35.103264    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:35.115402 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:35.115379    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:02:35.878389 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:02:35.878360    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:03:17.486156 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:17.486116    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:03:28.142007 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.141974    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"]
Jul 28 21:03:28.145633 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.145612    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.148086 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.148065    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-edb657-openshift-default-dockercfg-xhh5r\""
Jul 28 21:03:28.157483 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.157461    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"]
Jul 28 21:03:28.202416 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.202389    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-credential-socket\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.202525 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.202420    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56s5x\" (UniqueName: \"kubernetes.io/projected/ba145f84-a83d-4f5f-a127-86e8161b19ee-kube-api-access-56s5x\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.202525 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.202442    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-workload-certs\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.202525 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.202503    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba145f84-a83d-4f5f-a127-86e8161b19ee-istiod-ca-cert\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.202660 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.202538    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-workload-socket\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.202660 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.202631    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-data\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.202660 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.202658    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-token\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.202762 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.202682    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-podinfo\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.202762 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.202706    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-envoy\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.303282 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.303250    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-podinfo\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.303420 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.303286    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-envoy\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.303420 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.303311    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-credential-socket\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.303498 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.303421    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56s5x\" (UniqueName: \"kubernetes.io/projected/ba145f84-a83d-4f5f-a127-86e8161b19ee-kube-api-access-56s5x\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.303498 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.303467    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-workload-certs\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.303604 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.303521    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba145f84-a83d-4f5f-a127-86e8161b19ee-istiod-ca-cert\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.303653 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.303610    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-workload-socket\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.303719 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.303700    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-data\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.303776 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.303759    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-token\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.303830 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.303781    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-credential-socket\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.303894 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.303872    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-workload-certs\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.304148 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.304124    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-workload-socket\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.304240 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.304193    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-data\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.304341 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.304320    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba145f84-a83d-4f5f-a127-86e8161b19ee-istiod-ca-cert\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.305689 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.305658    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-envoy\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.305839 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.305822    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-podinfo\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.311270 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.311242    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-token\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.311359 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.311280    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56s5x\" (UniqueName: \"kubernetes.io/projected/ba145f84-a83d-4f5f-a127-86e8161b19ee-kube-api-access-56s5x\") pod \"e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") " pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.455911 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.455842    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:28.589055 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.589031    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"]
Jul 28 21:03:28.590971 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:03:28.590928    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba145f84_a83d_4f5f_a127_86e8161b19ee.slice/crio-c82648ec9ef89d0a6a9555e60f09fcdf694bae3bbf33269cf4201e13046aa800 WatchSource:0}: Error finding container c82648ec9ef89d0a6a9555e60f09fcdf694bae3bbf33269cf4201e13046aa800: Status 404 returned error can't find the container with id c82648ec9ef89d0a6a9555e60f09fcdf694bae3bbf33269cf4201e13046aa800
Jul 28 21:03:28.593184 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.593151    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 21:03:28.593270 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.593215    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 21:03:28.593270 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:28.593243    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 21:03:29.140963 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:29.140927    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd" event={"ID":"ba145f84-a83d-4f5f-a127-86e8161b19ee","Type":"ContainerStarted","Data":"14bd2964ddc61f958704f7e83d257b8d52b2f24f36272c11c02527f3faf1d5b9"}
Jul 28 21:03:29.141127 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:29.140982    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd" event={"ID":"ba145f84-a83d-4f5f-a127-86e8161b19ee","Type":"ContainerStarted","Data":"c82648ec9ef89d0a6a9555e60f09fcdf694bae3bbf33269cf4201e13046aa800"}
Jul 28 21:03:29.161754 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:29.161704    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd" podStartSLOduration=1.161684866 podStartE2EDuration="1.161684866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:03:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:03:29.160752458 +0000 UTC m=+3286.402845681" watchObservedRunningTime="2026-07-28 21:03:29.161684866 +0000 UTC m=+3286.403778089"
Jul 28 21:03:29.456430 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:29.456341    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:29.461108 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:29.461084    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:30.145892 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:30.145861    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:30.146760 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:30.146737    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:35.191150 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:35.191117    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:03:43.496062 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:43.496033    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 21:03:43.508752 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:43.508729    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"
Jul 28 21:03:44.228948 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:44.228847    2558 generic.go:356] "Generic (PLEG): container finished" podID="ba145f84-a83d-4f5f-a127-86e8161b19ee" containerID="14bd2964ddc61f958704f7e83d257b8d52b2f24f36272c11c02527f3faf1d5b9" exitCode=0
Jul 28 21:03:44.228948 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:44.228896    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd" event={"ID":"ba145f84-a83d-4f5f-a127-86e8161b19ee","Type":"ContainerDied","Data":"14bd2964ddc61f958704f7e83d257b8d52b2f24f36272c11c02527f3faf1d5b9"}
Jul 28 21:03:44.229386 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:44.229371    2558 scope.go:117] "RemoveContainer" containerID="14bd2964ddc61f958704f7e83d257b8d52b2f24f36272c11c02527f3faf1d5b9"
Jul 28 21:03:44.230493 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:44.230464    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 21:03:44.230613 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:44.230538    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 21:03:44.230613 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:44.230589    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 21:03:45.146088 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:45.146040    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:45.239707 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:45.239663    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd" event={"ID":"ba145f84-a83d-4f5f-a127-86e8161b19ee","Type":"ContainerStarted","Data":"9971f8d0db0eb8cc39963cf841144ec0190d430e5cc400c4e2145e086aa65cef"}
Jul 28 21:03:45.456617 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:45.456514    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:45.457776 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:45.457753    2558 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 28 21:03:45.457875 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:45.457800    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd" podUID="ba145f84-a83d-4f5f-a127-86e8161b19ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 28 21:03:46.456374 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:46.456339    2558 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 28 21:03:46.456777 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:46.456392    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd" podUID="ba145f84-a83d-4f5f-a127-86e8161b19ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 28 21:03:47.456821 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:47.456782    2558 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 28 21:03:47.457324 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:47.456886    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd" podUID="ba145f84-a83d-4f5f-a127-86e8161b19ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 28 21:03:48.456390 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:48.456358    2558 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 28 21:03:48.456619 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:48.456414    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd" podUID="ba145f84-a83d-4f5f-a127-86e8161b19ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 28 21:03:49.460133 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:49.460103    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:49.460550 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:49.460374    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:49.461173 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:49.461157    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:03:56.828444 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:56.828403    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c"]
Jul 28 21:03:56.830980 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:56.830961    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c"
Jul 28 21:03:56.833515 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:56.833490    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gcrkz\""
Jul 28 21:03:56.833515 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:56.833505    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 21:03:56.839496 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:56.839473    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c"]
Jul 28 21:03:56.981290 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:56.981260    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c01ebbe5-25da-4390-83a1-1556a49cfee4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c\" (UID: \"c01ebbe5-25da-4390-83a1-1556a49cfee4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c"
Jul 28 21:03:57.082096 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:57.082025    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c01ebbe5-25da-4390-83a1-1556a49cfee4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c\" (UID: \"c01ebbe5-25da-4390-83a1-1556a49cfee4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c"
Jul 28 21:03:57.082798 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:57.082770    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c01ebbe5-25da-4390-83a1-1556a49cfee4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c\" (UID: \"c01ebbe5-25da-4390-83a1-1556a49cfee4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c"
Jul 28 21:03:57.144391 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:57.144362    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c"
Jul 28 21:03:57.279948 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:57.279913    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c"]
Jul 28 21:03:57.306384 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:57.306346    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c" event={"ID":"c01ebbe5-25da-4390-83a1-1556a49cfee4","Type":"ContainerStarted","Data":"ab702decc644752d8ed7c6a4b90900b15fcb8cf9634c0121d314b79e5e75a2ed"}
Jul 28 21:03:58.313506 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:58.313475    2558 generic.go:356] "Generic (PLEG): container finished" podID="c01ebbe5-25da-4390-83a1-1556a49cfee4" containerID="af919cf34e2077bf88b07075422dce09ef94d295522cac1b36fd7ad46c5fa3d9" exitCode=0
Jul 28 21:03:58.313913 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:58.313589    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c" event={"ID":"c01ebbe5-25da-4390-83a1-1556a49cfee4","Type":"ContainerDied","Data":"af919cf34e2077bf88b07075422dce09ef94d295522cac1b36fd7ad46c5fa3d9"}
Jul 28 21:03:59.451017 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:59.450993    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c"
Jul 28 21:03:59.603250 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:59.603167    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c01ebbe5-25da-4390-83a1-1556a49cfee4-maas-api-service-ca\") pod \"c01ebbe5-25da-4390-83a1-1556a49cfee4\" (UID: \"c01ebbe5-25da-4390-83a1-1556a49cfee4\") "
Jul 28 21:03:59.603493 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:59.603470    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c01ebbe5-25da-4390-83a1-1556a49cfee4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c01ebbe5-25da-4390-83a1-1556a49cfee4" (UID: "c01ebbe5-25da-4390-83a1-1556a49cfee4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:03:59.704290 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:03:59.704261    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c01ebbe5-25da-4390-83a1-1556a49cfee4-maas-api-service-ca\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:00.325178 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:00.325147    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c"
Jul 28 21:04:00.325342 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:00.325180    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c" event={"ID":"c01ebbe5-25da-4390-83a1-1556a49cfee4","Type":"ContainerDied","Data":"ab702decc644752d8ed7c6a4b90900b15fcb8cf9634c0121d314b79e5e75a2ed"}
Jul 28 21:04:00.325342 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:00.325227    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab702decc644752d8ed7c6a4b90900b15fcb8cf9634c0121d314b79e5e75a2ed"
Jul 28 21:04:02.086090 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:02.086057    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c"]
Jul 28 21:04:02.088195 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:02.088162    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-edb657-zts9c"]
Jul 28 21:04:03.340425 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:03.340389    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c01ebbe5-25da-4390-83a1-1556a49cfee4" path="/var/lib/kubelet/pods/c01ebbe5-25da-4390-83a1-1556a49cfee4/volumes"
Jul 28 21:04:06.591469 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:06.591374    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:04:08.765959 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:08.765921    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"]
Jul 28 21:04:08.766349 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:08.766130    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd" podUID="ba145f84-a83d-4f5f-a127-86e8161b19ee" containerName="istio-proxy" containerID="cri-o://9971f8d0db0eb8cc39963cf841144ec0190d430e5cc400c4e2145e086aa65cef" gracePeriod=30
Jul 28 21:04:14.009855 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.009829    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:04:14.030086 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.030028    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-workload-socket\") pod \"ba145f84-a83d-4f5f-a127-86e8161b19ee\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") "
Jul 28 21:04:14.030086 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.030068    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-envoy\") pod \"ba145f84-a83d-4f5f-a127-86e8161b19ee\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") "
Jul 28 21:04:14.030263 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.030108    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56s5x\" (UniqueName: \"kubernetes.io/projected/ba145f84-a83d-4f5f-a127-86e8161b19ee-kube-api-access-56s5x\") pod \"ba145f84-a83d-4f5f-a127-86e8161b19ee\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") "
Jul 28 21:04:14.030263 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.030167    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba145f84-a83d-4f5f-a127-86e8161b19ee-istiod-ca-cert\") pod \"ba145f84-a83d-4f5f-a127-86e8161b19ee\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") "
Jul 28 21:04:14.030263 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.030207    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-credential-socket\") pod \"ba145f84-a83d-4f5f-a127-86e8161b19ee\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") "
Jul 28 21:04:14.030417 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.030265    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-podinfo\") pod \"ba145f84-a83d-4f5f-a127-86e8161b19ee\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") "
Jul 28 21:04:14.030833 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.030519    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-token\") pod \"ba145f84-a83d-4f5f-a127-86e8161b19ee\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") "
Jul 28 21:04:14.030833 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.030585    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-workload-certs\") pod \"ba145f84-a83d-4f5f-a127-86e8161b19ee\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") "
Jul 28 21:04:14.030833 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.030616    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-data\") pod \"ba145f84-a83d-4f5f-a127-86e8161b19ee\" (UID: \"ba145f84-a83d-4f5f-a127-86e8161b19ee\") "
Jul 28 21:04:14.031492 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.031418    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba145f84-a83d-4f5f-a127-86e8161b19ee" (UID: "ba145f84-a83d-4f5f-a127-86e8161b19ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:04:14.031492 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.031462    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba145f84-a83d-4f5f-a127-86e8161b19ee" (UID: "ba145f84-a83d-4f5f-a127-86e8161b19ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:04:14.031702 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.031502    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba145f84-a83d-4f5f-a127-86e8161b19ee" (UID: "ba145f84-a83d-4f5f-a127-86e8161b19ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:04:14.031819 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.031677    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba145f84-a83d-4f5f-a127-86e8161b19ee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba145f84-a83d-4f5f-a127-86e8161b19ee" (UID: "ba145f84-a83d-4f5f-a127-86e8161b19ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:04:14.032432 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.032374    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba145f84-a83d-4f5f-a127-86e8161b19ee" (UID: "ba145f84-a83d-4f5f-a127-86e8161b19ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:04:14.034178 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.034142    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba145f84-a83d-4f5f-a127-86e8161b19ee-kube-api-access-56s5x" (OuterVolumeSpecName: "kube-api-access-56s5x") pod "ba145f84-a83d-4f5f-a127-86e8161b19ee" (UID: "ba145f84-a83d-4f5f-a127-86e8161b19ee"). InnerVolumeSpecName "kube-api-access-56s5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:04:14.034374 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.034348    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba145f84-a83d-4f5f-a127-86e8161b19ee" (UID: "ba145f84-a83d-4f5f-a127-86e8161b19ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:04:14.034489 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.034388    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba145f84-a83d-4f5f-a127-86e8161b19ee" (UID: "ba145f84-a83d-4f5f-a127-86e8161b19ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:04:14.034489 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.034405    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba145f84-a83d-4f5f-a127-86e8161b19ee" (UID: "ba145f84-a83d-4f5f-a127-86e8161b19ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 21:04:14.131978 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.131953    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba145f84-a83d-4f5f-a127-86e8161b19ee-istiod-ca-cert\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:14.131978 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.131978    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-credential-socket\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:14.132134 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.131991    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-podinfo\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:14.132134 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.132005    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-token\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:14.132134 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.132016    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-workload-certs\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:14.132134 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.132027    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-data\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:14.132134 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.132041    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-workload-socket\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:14.132134 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.132054    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba145f84-a83d-4f5f-a127-86e8161b19ee-istio-envoy\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:14.132134 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.132066    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56s5x\" (UniqueName: \"kubernetes.io/projected/ba145f84-a83d-4f5f-a127-86e8161b19ee-kube-api-access-56s5x\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:14.400011 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.399933    2558 generic.go:356] "Generic (PLEG): container finished" podID="ba145f84-a83d-4f5f-a127-86e8161b19ee" containerID="9971f8d0db0eb8cc39963cf841144ec0190d430e5cc400c4e2145e086aa65cef" exitCode=0
Jul 28 21:04:14.400011 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.399983    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"
Jul 28 21:04:14.400165 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.400027    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd" event={"ID":"ba145f84-a83d-4f5f-a127-86e8161b19ee","Type":"ContainerDied","Data":"9971f8d0db0eb8cc39963cf841144ec0190d430e5cc400c4e2145e086aa65cef"}
Jul 28 21:04:14.400165 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.400092    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd" event={"ID":"ba145f84-a83d-4f5f-a127-86e8161b19ee","Type":"ContainerDied","Data":"c82648ec9ef89d0a6a9555e60f09fcdf694bae3bbf33269cf4201e13046aa800"}
Jul 28 21:04:14.400165 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.400119    2558 scope.go:117] "RemoveContainer" containerID="9971f8d0db0eb8cc39963cf841144ec0190d430e5cc400c4e2145e086aa65cef"
Jul 28 21:04:14.410014 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.409999    2558 scope.go:117] "RemoveContainer" containerID="14bd2964ddc61f958704f7e83d257b8d52b2f24f36272c11c02527f3faf1d5b9"
Jul 28 21:04:14.418326 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.418307    2558 scope.go:117] "RemoveContainer" containerID="9971f8d0db0eb8cc39963cf841144ec0190d430e5cc400c4e2145e086aa65cef"
Jul 28 21:04:14.418623 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:04:14.418603    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9971f8d0db0eb8cc39963cf841144ec0190d430e5cc400c4e2145e086aa65cef\": container with ID starting with 9971f8d0db0eb8cc39963cf841144ec0190d430e5cc400c4e2145e086aa65cef not found: ID does not exist" containerID="9971f8d0db0eb8cc39963cf841144ec0190d430e5cc400c4e2145e086aa65cef"
Jul 28 21:04:14.418685 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.418630    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9971f8d0db0eb8cc39963cf841144ec0190d430e5cc400c4e2145e086aa65cef"} err="failed to get container status \"9971f8d0db0eb8cc39963cf841144ec0190d430e5cc400c4e2145e086aa65cef\": rpc error: code = NotFound desc = could not find container \"9971f8d0db0eb8cc39963cf841144ec0190d430e5cc400c4e2145e086aa65cef\": container with ID starting with 9971f8d0db0eb8cc39963cf841144ec0190d430e5cc400c4e2145e086aa65cef not found: ID does not exist"
Jul 28 21:04:14.418685 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.418649    2558 scope.go:117] "RemoveContainer" containerID="14bd2964ddc61f958704f7e83d257b8d52b2f24f36272c11c02527f3faf1d5b9"
Jul 28 21:04:14.418913 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:04:14.418895    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14bd2964ddc61f958704f7e83d257b8d52b2f24f36272c11c02527f3faf1d5b9\": container with ID starting with 14bd2964ddc61f958704f7e83d257b8d52b2f24f36272c11c02527f3faf1d5b9 not found: ID does not exist" containerID="14bd2964ddc61f958704f7e83d257b8d52b2f24f36272c11c02527f3faf1d5b9"
Jul 28 21:04:14.418973 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.418922    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14bd2964ddc61f958704f7e83d257b8d52b2f24f36272c11c02527f3faf1d5b9"} err="failed to get container status \"14bd2964ddc61f958704f7e83d257b8d52b2f24f36272c11c02527f3faf1d5b9\": rpc error: code = NotFound desc = could not find container \"14bd2964ddc61f958704f7e83d257b8d52b2f24f36272c11c02527f3faf1d5b9\": container with ID starting with 14bd2964ddc61f958704f7e83d257b8d52b2f24f36272c11c02527f3faf1d5b9 not found: ID does not exist"
Jul 28 21:04:14.423479 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.423455    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"]
Jul 28 21:04:14.429197 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:14.429173    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-edb657-openshift-default-5b7487fd98-dszqd"]
Jul 28 21:04:15.340373 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:15.340331    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba145f84-a83d-4f5f-a127-86e8161b19ee" path="/var/lib/kubelet/pods/ba145f84-a83d-4f5f-a127-86e8161b19ee/volumes"
Jul 28 21:04:26.681742 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:26.681712    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:04:33.528802 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:33.528766    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"]
Jul 28 21:04:37.380106 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:37.380070    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:04:48.547543 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.547503    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8" podUID="eb1d11d7-4c1d-4b88-9516-8ecd51d72f00" containerName="main" containerID="cri-o://5122ffb90f3881164dec4a4078607a81aec719749885672a42fd98ce3045c822" gracePeriod=45
Jul 28 21:04:48.795067 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.795045    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:04:48.939694 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.939611    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-tls-certs\") pod \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") "
Jul 28 21:04:48.939694 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.939646    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-home\") pod \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") "
Jul 28 21:04:48.939694 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.939690    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjbjq\" (UniqueName: \"kubernetes.io/projected/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-kube-api-access-qjbjq\") pod \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") "
Jul 28 21:04:48.939967 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.939757    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-tmp-dir\") pod \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") "
Jul 28 21:04:48.939967 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.939791    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-model-cache\") pod \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") "
Jul 28 21:04:48.939967 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.939848    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-dshm\") pod \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\" (UID: \"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00\") "
Jul 28 21:04:48.939967 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.939894    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-home" (OuterVolumeSpecName: "home") pod "eb1d11d7-4c1d-4b88-9516-8ecd51d72f00" (UID: "eb1d11d7-4c1d-4b88-9516-8ecd51d72f00"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:04:48.940169 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.940085    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eb1d11d7-4c1d-4b88-9516-8ecd51d72f00" (UID: "eb1d11d7-4c1d-4b88-9516-8ecd51d72f00"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:04:48.940169 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.940092    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-model-cache" (OuterVolumeSpecName: "model-cache") pod "eb1d11d7-4c1d-4b88-9516-8ecd51d72f00" (UID: "eb1d11d7-4c1d-4b88-9516-8ecd51d72f00"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:04:48.940337 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.940203    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-tmp-dir\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:48.940337 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.940217    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-model-cache\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:48.940337 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.940226    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-home\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:48.941986 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.941954    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-dshm" (OuterVolumeSpecName: "dshm") pod "eb1d11d7-4c1d-4b88-9516-8ecd51d72f00" (UID: "eb1d11d7-4c1d-4b88-9516-8ecd51d72f00"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:04:48.941986 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.941983    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-kube-api-access-qjbjq" (OuterVolumeSpecName: "kube-api-access-qjbjq") pod "eb1d11d7-4c1d-4b88-9516-8ecd51d72f00" (UID: "eb1d11d7-4c1d-4b88-9516-8ecd51d72f00"). InnerVolumeSpecName "kube-api-access-qjbjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:04:48.942141 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:48.942023    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eb1d11d7-4c1d-4b88-9516-8ecd51d72f00" (UID: "eb1d11d7-4c1d-4b88-9516-8ecd51d72f00"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:04:49.041034 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:49.041011    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-dshm\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:49.041034 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:49.041031    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-tls-certs\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:49.041175 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:49.041042    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjbjq\" (UniqueName: \"kubernetes.io/projected/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00-kube-api-access-qjbjq\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:04:49.590706 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:49.590672    2558 generic.go:356] "Generic (PLEG): container finished" podID="eb1d11d7-4c1d-4b88-9516-8ecd51d72f00" containerID="5122ffb90f3881164dec4a4078607a81aec719749885672a42fd98ce3045c822" exitCode=0
Jul 28 21:04:49.591149 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:49.590727    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"
Jul 28 21:04:49.591149 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:49.590759    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8" event={"ID":"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00","Type":"ContainerDied","Data":"5122ffb90f3881164dec4a4078607a81aec719749885672a42fd98ce3045c822"}
Jul 28 21:04:49.591149 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:49.590811    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8" event={"ID":"eb1d11d7-4c1d-4b88-9516-8ecd51d72f00","Type":"ContainerDied","Data":"5af60a3222ed4ba4feb10d615d9c8b0c21330a2ec383b6e2616d76b6a61d4bd3"}
Jul 28 21:04:49.591149 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:49.590833    2558 scope.go:117] "RemoveContainer" containerID="5122ffb90f3881164dec4a4078607a81aec719749885672a42fd98ce3045c822"
Jul 28 21:04:49.600256 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:49.600234    2558 scope.go:117] "RemoveContainer" containerID="5122ffb90f3881164dec4a4078607a81aec719749885672a42fd98ce3045c822"
Jul 28 21:04:49.600546 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:04:49.600515    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5122ffb90f3881164dec4a4078607a81aec719749885672a42fd98ce3045c822\": container with ID starting with 5122ffb90f3881164dec4a4078607a81aec719749885672a42fd98ce3045c822 not found: ID does not exist" containerID="5122ffb90f3881164dec4a4078607a81aec719749885672a42fd98ce3045c822"
Jul 28 21:04:49.600672 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:49.600618    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5122ffb90f3881164dec4a4078607a81aec719749885672a42fd98ce3045c822"} err="failed to get container status \"5122ffb90f3881164dec4a4078607a81aec719749885672a42fd98ce3045c822\": rpc error: code = NotFound desc = could not find container \"5122ffb90f3881164dec4a4078607a81aec719749885672a42fd98ce3045c822\": container with ID starting with 5122ffb90f3881164dec4a4078607a81aec719749885672a42fd98ce3045c822 not found: ID does not exist"
Jul 28 21:04:49.608372 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:49.608351    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"]
Jul 28 21:04:49.612726 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:49.612706    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-ce337a/ipp-route-ce337a-kserve-5d84c678c5-7wwj8"]
Jul 28 21:04:51.340958 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:04:51.340921    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb1d11d7-4c1d-4b88-9516-8ecd51d72f00" path="/var/lib/kubelet/pods/eb1d11d7-4c1d-4b88-9516-8ecd51d72f00/volumes"
Jul 28 21:05:17.105572 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.105523    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7"]
Jul 28 21:05:17.106118 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.106100    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb1d11d7-4c1d-4b88-9516-8ecd51d72f00" containerName="main"
Jul 28 21:05:17.106162 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.106121    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb1d11d7-4c1d-4b88-9516-8ecd51d72f00" containerName="main"
Jul 28 21:05:17.106162 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.106135    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba145f84-a83d-4f5f-a127-86e8161b19ee" containerName="istio-proxy"
Jul 28 21:05:17.106162 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.106143    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba145f84-a83d-4f5f-a127-86e8161b19ee" containerName="istio-proxy"
Jul 28 21:05:17.106162 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.106154    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba145f84-a83d-4f5f-a127-86e8161b19ee" containerName="istio-proxy"
Jul 28 21:05:17.106162 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.106163    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba145f84-a83d-4f5f-a127-86e8161b19ee" containerName="istio-proxy"
Jul 28 21:05:17.106320 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.106174    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c01ebbe5-25da-4390-83a1-1556a49cfee4" containerName="revoke-keys"
Jul 28 21:05:17.106320 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.106182    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c01ebbe5-25da-4390-83a1-1556a49cfee4" containerName="revoke-keys"
Jul 28 21:05:17.106320 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.106300    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba145f84-a83d-4f5f-a127-86e8161b19ee" containerName="istio-proxy"
Jul 28 21:05:17.106320 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.106309    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba145f84-a83d-4f5f-a127-86e8161b19ee" containerName="istio-proxy"
Jul 28 21:05:17.106320 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.106317    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb1d11d7-4c1d-4b88-9516-8ecd51d72f00" containerName="main"
Jul 28 21:05:17.106468 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.106327    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c01ebbe5-25da-4390-83a1-1556a49cfee4" containerName="revoke-keys"
Jul 28 21:05:17.109872 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.109854    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7"
Jul 28 21:05:17.113260 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.113237    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 21:05:17.113398 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.113279    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gcrkz\""
Jul 28 21:05:17.117663 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.117633    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7"]
Jul 28 21:05:17.175152 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.175109    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b701d8b4-3210-4f8d-9e74-8d8f10758c46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-542599-vglr7\" (UID: \"b701d8b4-3210-4f8d-9e74-8d8f10758c46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7"
Jul 28 21:05:17.275690 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.275661    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b701d8b4-3210-4f8d-9e74-8d8f10758c46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-542599-vglr7\" (UID: \"b701d8b4-3210-4f8d-9e74-8d8f10758c46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7"
Jul 28 21:05:17.276211 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.276193    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b701d8b4-3210-4f8d-9e74-8d8f10758c46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-542599-vglr7\" (UID: \"b701d8b4-3210-4f8d-9e74-8d8f10758c46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7"
Jul 28 21:05:17.423550 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.423480    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7"
Jul 28 21:05:17.761754 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:17.761729    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7"]
Jul 28 21:05:17.764071 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:05:17.764043    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb701d8b4_3210_4f8d_9e74_8d8f10758c46.slice/crio-a606ad10e02619434468e7f14096f5e167f1adfd81bd44a156c5c0868ece58a0 WatchSource:0}: Error finding container a606ad10e02619434468e7f14096f5e167f1adfd81bd44a156c5c0868ece58a0: Status 404 returned error can't find the container with id a606ad10e02619434468e7f14096f5e167f1adfd81bd44a156c5c0868ece58a0
Jul 28 21:05:18.748461 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:18.748384    2558 generic.go:356] "Generic (PLEG): container finished" podID="b701d8b4-3210-4f8d-9e74-8d8f10758c46" containerID="21c026990127539452dfd5c2d25c942eb85216ad0d55dc52de59720cb2ab6e64" exitCode=0
Jul 28 21:05:18.748461 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:18.748449    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7" event={"ID":"b701d8b4-3210-4f8d-9e74-8d8f10758c46","Type":"ContainerDied","Data":"21c026990127539452dfd5c2d25c942eb85216ad0d55dc52de59720cb2ab6e64"}
Jul 28 21:05:18.748858 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:18.748487    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7" event={"ID":"b701d8b4-3210-4f8d-9e74-8d8f10758c46","Type":"ContainerStarted","Data":"a606ad10e02619434468e7f14096f5e167f1adfd81bd44a156c5c0868ece58a0"}
Jul 28 21:05:19.878135 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:19.878112    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7"
Jul 28 21:05:19.997156 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:19.997119    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b701d8b4-3210-4f8d-9e74-8d8f10758c46-maas-api-service-ca\") pod \"b701d8b4-3210-4f8d-9e74-8d8f10758c46\" (UID: \"b701d8b4-3210-4f8d-9e74-8d8f10758c46\") "
Jul 28 21:05:19.997444 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:19.997422    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b701d8b4-3210-4f8d-9e74-8d8f10758c46-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b701d8b4-3210-4f8d-9e74-8d8f10758c46" (UID: "b701d8b4-3210-4f8d-9e74-8d8f10758c46"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:05:20.098583 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:20.098498    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b701d8b4-3210-4f8d-9e74-8d8f10758c46-maas-api-service-ca\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:05:20.762712 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:20.762689    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7"
Jul 28 21:05:20.762880 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:20.762724    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7" event={"ID":"b701d8b4-3210-4f8d-9e74-8d8f10758c46","Type":"ContainerDied","Data":"a606ad10e02619434468e7f14096f5e167f1adfd81bd44a156c5c0868ece58a0"}
Jul 28 21:05:20.762880 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:20.762770    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a606ad10e02619434468e7f14096f5e167f1adfd81bd44a156c5c0868ece58a0"
Jul 28 21:05:28.452050 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:28.452006    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7"]
Jul 28 21:05:28.454609 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:28.454585    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-542599-vglr7"]
Jul 28 21:05:29.340012 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:29.339979    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b701d8b4-3210-4f8d-9e74-8d8f10758c46" path="/var/lib/kubelet/pods/b701d8b4-3210-4f8d-9e74-8d8f10758c46/volumes"
Jul 28 21:05:31.284980 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:31.284946    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:05:34.992667 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:34.992587    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"]
Jul 28 21:05:34.993072 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:34.992857    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5" podUID="009b6a70-90cb-4c9e-8903-a0ff2d2b654f" containerName="istio-proxy" containerID="cri-o://b375eac2c2453a629e59d0525d76467badeea060d5a959689fd93fb4c33a3e01" gracePeriod=30
Jul 28 21:05:35.428856 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:35.428827    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg"]
Jul 28 21:05:35.429223 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:35.429209    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b701d8b4-3210-4f8d-9e74-8d8f10758c46" containerName="revoke-keys"
Jul 28 21:05:35.429268 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:35.429225    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b701d8b4-3210-4f8d-9e74-8d8f10758c46" containerName="revoke-keys"
Jul 28 21:05:35.429307 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:35.429299    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b701d8b4-3210-4f8d-9e74-8d8f10758c46" containerName="revoke-keys"
Jul 28 21:05:35.432479 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:35.432460    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg"
Jul 28 21:05:35.435087 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:35.435065    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gcrkz\""
Jul 28 21:05:35.435216 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:35.435142    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 21:05:35.440133 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:35.439949    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg"]
Jul 28 21:05:35.528351 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:35.528328    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5093578-dddd-45e6-8d63-00072d9605cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg\" (UID: \"c5093578-dddd-45e6-8d63-00072d9605cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg"
Jul 28 21:05:35.629122 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:35.629095    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5093578-dddd-45e6-8d63-00072d9605cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg\" (UID: \"c5093578-dddd-45e6-8d63-00072d9605cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg"
Jul 28 21:05:35.629699 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:35.629681    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5093578-dddd-45e6-8d63-00072d9605cf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg\" (UID: \"c5093578-dddd-45e6-8d63-00072d9605cf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg"
Jul 28 21:05:35.744547 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:35.744485    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg"
Jul 28 21:05:35.872870 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:35.872840    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg"]
Jul 28 21:05:35.874449 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:05:35.874424    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5093578_dddd_45e6_8d63_00072d9605cf.slice/crio-1ddc0e97408bd111d4b1c38e50b41599bce4c27a1e8b640f7b1f1eb5f3939f2a WatchSource:0}: Error finding container 1ddc0e97408bd111d4b1c38e50b41599bce4c27a1e8b640f7b1f1eb5f3939f2a: Status 404 returned error can't find the container with id 1ddc0e97408bd111d4b1c38e50b41599bce4c27a1e8b640f7b1f1eb5f3939f2a
Jul 28 21:05:36.851744 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:36.851714    2558 generic.go:356] "Generic (PLEG): container finished" podID="c5093578-dddd-45e6-8d63-00072d9605cf" containerID="7b8fef826546819ba6b8efde41c39990676c9a2f338b989407a63c5792fdc306" exitCode=0
Jul 28 21:05:36.852093 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:36.851755    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg" event={"ID":"c5093578-dddd-45e6-8d63-00072d9605cf","Type":"ContainerDied","Data":"7b8fef826546819ba6b8efde41c39990676c9a2f338b989407a63c5792fdc306"}
Jul 28 21:05:36.852093 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:36.851813    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg" event={"ID":"c5093578-dddd-45e6-8d63-00072d9605cf","Type":"ContainerStarted","Data":"1ddc0e97408bd111d4b1c38e50b41599bce4c27a1e8b640f7b1f1eb5f3939f2a"}
Jul 28 21:05:37.994390 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:37.994368    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg"
Jul 28 21:05:38.152642 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:38.152573    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5093578-dddd-45e6-8d63-00072d9605cf-maas-api-service-ca\") pod \"c5093578-dddd-45e6-8d63-00072d9605cf\" (UID: \"c5093578-dddd-45e6-8d63-00072d9605cf\") "
Jul 28 21:05:38.152849 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:38.152829    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5093578-dddd-45e6-8d63-00072d9605cf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5093578-dddd-45e6-8d63-00072d9605cf" (UID: "c5093578-dddd-45e6-8d63-00072d9605cf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:05:38.253600 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:38.253575    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5093578-dddd-45e6-8d63-00072d9605cf-maas-api-service-ca\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:05:38.863845 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:38.863811    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg"
Jul 28 21:05:38.864029 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:38.863851    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg" event={"ID":"c5093578-dddd-45e6-8d63-00072d9605cf","Type":"ContainerDied","Data":"1ddc0e97408bd111d4b1c38e50b41599bce4c27a1e8b640f7b1f1eb5f3939f2a"}
Jul 28 21:05:38.864029 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:38.863912    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ddc0e97408bd111d4b1c38e50b41599bce4c27a1e8b640f7b1f1eb5f3939f2a"
Jul 28 21:05:40.258670 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.258612    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:05:40.370536 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.370497    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-token\") pod \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") "
Jul 28 21:05:40.370703 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.370573    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istiod-ca-cert\") pod \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") "
Jul 28 21:05:40.370703 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.370604    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-data\") pod \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") "
Jul 28 21:05:40.370703 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.370650    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-podinfo\") pod \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") "
Jul 28 21:05:40.370703 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.370677    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6tgx\" (UniqueName: \"kubernetes.io/projected/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-kube-api-access-r6tgx\") pod \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") "
Jul 28 21:05:40.370885 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.370712    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-credential-socket\") pod \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") "
Jul 28 21:05:40.370885 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.370733    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-envoy\") pod \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") "
Jul 28 21:05:40.370885 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.370768    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-workload-certs\") pod \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") "
Jul 28 21:05:40.370885 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.370824    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-workload-socket\") pod \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\" (UID: \"009b6a70-90cb-4c9e-8903-a0ff2d2b654f\") "
Jul 28 21:05:40.371097 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.371054    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "009b6a70-90cb-4c9e-8903-a0ff2d2b654f" (UID: "009b6a70-90cb-4c9e-8903-a0ff2d2b654f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:05:40.371150 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.371090    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "009b6a70-90cb-4c9e-8903-a0ff2d2b654f" (UID: "009b6a70-90cb-4c9e-8903-a0ff2d2b654f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:05:40.371236 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.371188    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istiod-ca-cert\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:05:40.371312 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.371241    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-credential-socket\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:05:40.371312 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.371247    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "009b6a70-90cb-4c9e-8903-a0ff2d2b654f" (UID: "009b6a70-90cb-4c9e-8903-a0ff2d2b654f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:05:40.371380 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.371342    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "009b6a70-90cb-4c9e-8903-a0ff2d2b654f" (UID: "009b6a70-90cb-4c9e-8903-a0ff2d2b654f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:05:40.371652 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.371625    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-data" (OuterVolumeSpecName: "istio-data") pod "009b6a70-90cb-4c9e-8903-a0ff2d2b654f" (UID: "009b6a70-90cb-4c9e-8903-a0ff2d2b654f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:05:40.372867 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.372835    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-token" (OuterVolumeSpecName: "istio-token") pod "009b6a70-90cb-4c9e-8903-a0ff2d2b654f" (UID: "009b6a70-90cb-4c9e-8903-a0ff2d2b654f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:05:40.372997 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.372880    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "009b6a70-90cb-4c9e-8903-a0ff2d2b654f" (UID: "009b6a70-90cb-4c9e-8903-a0ff2d2b654f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 21:05:40.372997 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.372977    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "009b6a70-90cb-4c9e-8903-a0ff2d2b654f" (UID: "009b6a70-90cb-4c9e-8903-a0ff2d2b654f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 21:05:40.373116 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.373066    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-kube-api-access-r6tgx" (OuterVolumeSpecName: "kube-api-access-r6tgx") pod "009b6a70-90cb-4c9e-8903-a0ff2d2b654f" (UID: "009b6a70-90cb-4c9e-8903-a0ff2d2b654f"). InnerVolumeSpecName "kube-api-access-r6tgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:05:40.472140 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.472103    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-workload-socket\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:05:40.472263 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.472149    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-token\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:05:40.472263 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.472164    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-data\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:05:40.472263 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.472172    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-podinfo\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:05:40.472263 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.472192    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6tgx\" (UniqueName: \"kubernetes.io/projected/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-kube-api-access-r6tgx\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:05:40.472263 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.472208    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-istio-envoy\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:05:40.472263 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.472219    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/009b6a70-90cb-4c9e-8903-a0ff2d2b654f-workload-certs\") on node \"ip-10-0-133-72.ec2.internal\" DevicePath \"\""
Jul 28 21:05:40.880550 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.880511    2558 generic.go:356] "Generic (PLEG): container finished" podID="009b6a70-90cb-4c9e-8903-a0ff2d2b654f" containerID="b375eac2c2453a629e59d0525d76467badeea060d5a959689fd93fb4c33a3e01" exitCode=0
Jul 28 21:05:40.880739 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.880599    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"
Jul 28 21:05:40.880739 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.880599    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5" event={"ID":"009b6a70-90cb-4c9e-8903-a0ff2d2b654f","Type":"ContainerDied","Data":"b375eac2c2453a629e59d0525d76467badeea060d5a959689fd93fb4c33a3e01"}
Jul 28 21:05:40.880876 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.880757    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5" event={"ID":"009b6a70-90cb-4c9e-8903-a0ff2d2b654f","Type":"ContainerDied","Data":"714cbdea00231d2f1d619f9fdfb0657e4efbe59a55a377472979f1df4e68fb61"}
Jul 28 21:05:40.880876 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.880780    2558 scope.go:117] "RemoveContainer" containerID="b375eac2c2453a629e59d0525d76467badeea060d5a959689fd93fb4c33a3e01"
Jul 28 21:05:40.890832 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.890814    2558 scope.go:117] "RemoveContainer" containerID="0ae68ed746f4af8fb97b4639573b366cc81049a8074257fbaacc7db776844ab2"
Jul 28 21:05:40.898970 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.898955    2558 scope.go:117] "RemoveContainer" containerID="b375eac2c2453a629e59d0525d76467badeea060d5a959689fd93fb4c33a3e01"
Jul 28 21:05:40.899186 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:05:40.899169    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b375eac2c2453a629e59d0525d76467badeea060d5a959689fd93fb4c33a3e01\": container with ID starting with b375eac2c2453a629e59d0525d76467badeea060d5a959689fd93fb4c33a3e01 not found: ID does not exist" containerID="b375eac2c2453a629e59d0525d76467badeea060d5a959689fd93fb4c33a3e01"
Jul 28 21:05:40.899229 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.899194    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b375eac2c2453a629e59d0525d76467badeea060d5a959689fd93fb4c33a3e01"} err="failed to get container status \"b375eac2c2453a629e59d0525d76467badeea060d5a959689fd93fb4c33a3e01\": rpc error: code = NotFound desc = could not find container \"b375eac2c2453a629e59d0525d76467badeea060d5a959689fd93fb4c33a3e01\": container with ID starting with b375eac2c2453a629e59d0525d76467badeea060d5a959689fd93fb4c33a3e01 not found: ID does not exist"
Jul 28 21:05:40.899229 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.899210    2558 scope.go:117] "RemoveContainer" containerID="0ae68ed746f4af8fb97b4639573b366cc81049a8074257fbaacc7db776844ab2"
Jul 28 21:05:40.899406 ip-10-0-133-72 kubenswrapper[2558]: E0728 21:05:40.899390    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ae68ed746f4af8fb97b4639573b366cc81049a8074257fbaacc7db776844ab2\": container with ID starting with 0ae68ed746f4af8fb97b4639573b366cc81049a8074257fbaacc7db776844ab2 not found: ID does not exist" containerID="0ae68ed746f4af8fb97b4639573b366cc81049a8074257fbaacc7db776844ab2"
Jul 28 21:05:40.899445 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.899410    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ae68ed746f4af8fb97b4639573b366cc81049a8074257fbaacc7db776844ab2"} err="failed to get container status \"0ae68ed746f4af8fb97b4639573b366cc81049a8074257fbaacc7db776844ab2\": rpc error: code = NotFound desc = could not find container \"0ae68ed746f4af8fb97b4639573b366cc81049a8074257fbaacc7db776844ab2\": container with ID starting with 0ae68ed746f4af8fb97b4639573b366cc81049a8074257fbaacc7db776844ab2 not found: ID does not exist"
Jul 28 21:05:40.903366 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.903343    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"]
Jul 28 21:05:40.909286 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:40.909267    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-542599-openshift-default-7df7d88496-gh5p5"]
Jul 28 21:05:41.340062 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:41.340033    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="009b6a70-90cb-4c9e-8903-a0ff2d2b654f" path="/var/lib/kubelet/pods/009b6a70-90cb-4c9e-8903-a0ff2d2b654f/volumes"
Jul 28 21:05:45.854411 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:45.854381    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n"]
Jul 28 21:05:45.857510 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:45.857479    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-cc370c-29754540-bk29n"]
Jul 28 21:05:46.033321 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:46.033286    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg"]
Jul 28 21:05:46.035762 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:46.035733    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-cc370c-v2lrg"]
Jul 28 21:05:47.340832 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:47.340799    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b498bab-8b32-4065-9c54-59b53f1739ee" path="/var/lib/kubelet/pods/5b498bab-8b32-4065-9c54-59b53f1739ee/volumes"
Jul 28 21:05:47.341272 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:47.341084    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5093578-dddd-45e6-8d63-00072d9605cf" path="/var/lib/kubelet/pods/c5093578-dddd-45e6-8d63-00072d9605cf/volumes"
Jul 28 21:05:49.387569 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:05:49.387535    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:06:03.484736 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:03.484701    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx"]
Jul 28 21:06:03.489633 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:03.489607    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-35c841-29754540-qn5bx"]
Jul 28 21:06:05.340234 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:05.340200    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3de6d1d2-fed2-43f4-bcfd-4c5dd118260f" path="/var/lib/kubelet/pods/3de6d1d2-fed2-43f4-bcfd-4c5dd118260f/volumes"
Jul 28 21:06:06.387465 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:06.387431    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpzd2"]
Jul 28 21:06:10.779241 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:10.779196    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c987cb57b-gcgp7_b72fb21e-3843-49e2-af39-d038022fa1e7/authorino/0.log"
Jul 28 21:06:17.118898 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:17.118859    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74ff7fbd5d-kbj97_18c742c1-226f-4208-a374-e7cbec444d18/copy-manifests/0.log"
Jul 28 21:06:17.126461 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:17.126431    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74ff7fbd5d-kbj97_18c742c1-226f-4208-a374-e7cbec444d18/manager/0.log"
Jul 28 21:06:17.958203 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:17.958172    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754540-mgfpc_bb07ba0d-3b85-4ef6-8d42-cd23ba191b74/cleanup/0.log"
Jul 28 21:06:19.390305 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:19.390264    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c987cb57b-gcgp7_b72fb21e-3843-49e2-af39-d038022fa1e7/authorino/0.log"
Jul 28 21:06:19.757086 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:19.757058    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kzlwd_7f9febd1-7e8a-40a9-829f-4151279e95dc/kuadrant-console-plugin/0.log"
Jul 28 21:06:20.116278 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:20.116207    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-tpzd2_a17fc07b-65ac-4da7-b9d5-1dc67bb60f10/limitador/0.log"
Jul 28 21:06:20.962988 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:20.962960    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bb564c89-wtzwv_115f81c7-6e64-414a-aa77-fac6a2dc2224/kube-auth-proxy/0.log"
Jul 28 21:06:22.033574 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:22.033541    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-r2xzn_2fba53f0-8999-4d98-bb12-5d3e13cf657c/main/0.log"
Jul 28 21:06:22.385308 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:22.385198    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-2pvxb_5ec4a624-13bd-447d-8f96-f1c8c7030d29/main/0.log"
Jul 28 21:06:30.616475 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:30.616445    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zmkbq_e1b683c6-ec1a-4f37-b354-3a366d2bd899/global-pull-secret-syncer/0.log"
Jul 28 21:06:30.761105 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:30.761074    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w628c_2692fbe8-7cff-4492-99fc-a18edc6b24fa/konnectivity-agent/0.log"
Jul 28 21:06:30.785738 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:30.785716    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-72.ec2.internal_ada8b40b091a0bc1a7e4aff59957a1c4/haproxy/0.log"
Jul 28 21:06:36.007284 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:36.007256    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c987cb57b-gcgp7_b72fb21e-3843-49e2-af39-d038022fa1e7/authorino/0.log"
Jul 28 21:06:36.191506 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:36.191482    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kzlwd_7f9febd1-7e8a-40a9-829f-4151279e95dc/kuadrant-console-plugin/0.log"
Jul 28 21:06:36.503780 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:36.503747    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-tpzd2_a17fc07b-65ac-4da7-b9d5-1dc67bb60f10/limitador/0.log"
Jul 28 21:06:38.135921 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.135858    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6fc43d4a-cdec-432d-b0a6-131f8dd2642c/alertmanager/0.log"
Jul 28 21:06:38.158757 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.158735    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6fc43d4a-cdec-432d-b0a6-131f8dd2642c/config-reloader/0.log"
Jul 28 21:06:38.184643 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.184622    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6fc43d4a-cdec-432d-b0a6-131f8dd2642c/kube-rbac-proxy-web/0.log"
Jul 28 21:06:38.205574 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.205538    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6fc43d4a-cdec-432d-b0a6-131f8dd2642c/kube-rbac-proxy/0.log"
Jul 28 21:06:38.228117 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.228097    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6fc43d4a-cdec-432d-b0a6-131f8dd2642c/kube-rbac-proxy-metric/0.log"
Jul 28 21:06:38.249420 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.249397    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6fc43d4a-cdec-432d-b0a6-131f8dd2642c/prom-label-proxy/0.log"
Jul 28 21:06:38.273541 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.273521    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6fc43d4a-cdec-432d-b0a6-131f8dd2642c/init-config-reloader/0.log"
Jul 28 21:06:38.364769 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.364751    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-wxzsk_517464c7-0e99-4501-a815-8cf75872c95c/kube-state-metrics/0.log"
Jul 28 21:06:38.384640 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.384626    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-wxzsk_517464c7-0e99-4501-a815-8cf75872c95c/kube-rbac-proxy-main/0.log"
Jul 28 21:06:38.407251 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.407186    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-wxzsk_517464c7-0e99-4501-a815-8cf75872c95c/kube-rbac-proxy-self/0.log"
Jul 28 21:06:38.436706 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.436683    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-67bf6bb647-f58q9_c37412a1-56e7-4d17-aa4d-53d08090b1d0/metrics-server/0.log"
Jul 28 21:06:38.465320 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.465295    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-tw4j4_26107ef4-9268-4af8-aeba-72115376a2ae/monitoring-plugin/0.log"
Jul 28 21:06:38.674460 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.674392    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pl5jb_124e3837-d436-4036-8940-007ce3e88ab5/node-exporter/0.log"
Jul 28 21:06:38.698274 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.698257    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pl5jb_124e3837-d436-4036-8940-007ce3e88ab5/kube-rbac-proxy/0.log"
Jul 28 21:06:38.730409 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.730392    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pl5jb_124e3837-d436-4036-8940-007ce3e88ab5/init-textfile/0.log"
Jul 28 21:06:38.756932 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.756896    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-s5gr8_95abda6d-c7b6-4b65-87f6-f5a3af15f232/kube-rbac-proxy-main/0.log"
Jul 28 21:06:38.787123 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.787100    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-s5gr8_95abda6d-c7b6-4b65-87f6-f5a3af15f232/kube-rbac-proxy-self/0.log"
Jul 28 21:06:38.808670 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.808636    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-s5gr8_95abda6d-c7b6-4b65-87f6-f5a3af15f232/openshift-state-metrics/0.log"
Jul 28 21:06:38.846217 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.846191    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_921afb17-7890-44c5-81f9-01736a22bfae/prometheus/0.log"
Jul 28 21:06:38.868159 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.868141    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_921afb17-7890-44c5-81f9-01736a22bfae/config-reloader/0.log"
Jul 28 21:06:38.889151 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.889126    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_921afb17-7890-44c5-81f9-01736a22bfae/thanos-sidecar/0.log"
Jul 28 21:06:38.909659 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.909644    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_921afb17-7890-44c5-81f9-01736a22bfae/kube-rbac-proxy-web/0.log"
Jul 28 21:06:38.930689 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.930643    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_921afb17-7890-44c5-81f9-01736a22bfae/kube-rbac-proxy/0.log"
Jul 28 21:06:38.952945 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.952923    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_921afb17-7890-44c5-81f9-01736a22bfae/kube-rbac-proxy-thanos/0.log"
Jul 28 21:06:38.979217 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:38.979198    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_921afb17-7890-44c5-81f9-01736a22bfae/init-config-reloader/0.log"
Jul 28 21:06:39.004902 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:39.004883    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-wg7mf_1a87ca62-5860-4bde-80d9-600f1b9172d1/prometheus-operator/0.log"
Jul 28 21:06:39.022739 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:39.022721    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-wg7mf_1a87ca62-5860-4bde-80d9-600f1b9172d1/kube-rbac-proxy/0.log"
Jul 28 21:06:39.043746 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:39.043729    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-g9fxr_d58be913-bc73-4252-a974-5ebffbb9485a/prometheus-operator-admission-webhook/0.log"
Jul 28 21:06:39.081774 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:39.081757    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b65dfccdf-vvwkd_778eeadb-b700-411e-9367-8112cd14d750/telemeter-client/0.log"
Jul 28 21:06:39.109249 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:39.109231    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b65dfccdf-vvwkd_778eeadb-b700-411e-9367-8112cd14d750/reload/0.log"
Jul 28 21:06:39.131019 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:39.130992    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b65dfccdf-vvwkd_778eeadb-b700-411e-9367-8112cd14d750/kube-rbac-proxy/0.log"
Jul 28 21:06:39.163122 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:39.163103    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689b76fbb5-9jsq5_0f41c4be-2757-4d01-8117-3a6331c4d413/thanos-query/0.log"
Jul 28 21:06:39.187123 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:39.187076    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689b76fbb5-9jsq5_0f41c4be-2757-4d01-8117-3a6331c4d413/kube-rbac-proxy-web/0.log"
Jul 28 21:06:39.213501 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:39.213486    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689b76fbb5-9jsq5_0f41c4be-2757-4d01-8117-3a6331c4d413/kube-rbac-proxy/0.log"
Jul 28 21:06:39.235004 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:39.234987    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689b76fbb5-9jsq5_0f41c4be-2757-4d01-8117-3a6331c4d413/prom-label-proxy/0.log"
Jul 28 21:06:39.255348 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:39.255328    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689b76fbb5-9jsq5_0f41c4be-2757-4d01-8117-3a6331c4d413/kube-rbac-proxy-rules/0.log"
Jul 28 21:06:39.279065 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:39.279048    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689b76fbb5-9jsq5_0f41c4be-2757-4d01-8117-3a6331c4d413/kube-rbac-proxy-metrics/0.log"
Jul 28 21:06:40.253919 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.253887    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"]
Jul 28 21:06:40.254277 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.254254    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="009b6a70-90cb-4c9e-8903-a0ff2d2b654f" containerName="istio-proxy"
Jul 28 21:06:40.254277 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.254264    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="009b6a70-90cb-4c9e-8903-a0ff2d2b654f" containerName="istio-proxy"
Jul 28 21:06:40.254351 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.254280    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5093578-dddd-45e6-8d63-00072d9605cf" containerName="revoke-keys"
Jul 28 21:06:40.254351 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.254287    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5093578-dddd-45e6-8d63-00072d9605cf" containerName="revoke-keys"
Jul 28 21:06:40.254351 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.254347    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5093578-dddd-45e6-8d63-00072d9605cf" containerName="revoke-keys"
Jul 28 21:06:40.254445 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.254358    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="009b6a70-90cb-4c9e-8903-a0ff2d2b654f" containerName="istio-proxy"
Jul 28 21:06:40.254445 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.254364    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="009b6a70-90cb-4c9e-8903-a0ff2d2b654f" containerName="istio-proxy"
Jul 28 21:06:40.257277 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.257260    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.259671 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.259650    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cm86f\"/\"default-dockercfg-xkx86\""
Jul 28 21:06:40.259874 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.259860    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cm86f\"/\"openshift-service-ca.crt\""
Jul 28 21:06:40.260651 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.260631    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cm86f\"/\"kube-root-ca.crt\""
Jul 28 21:06:40.266767 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.266743    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"]
Jul 28 21:06:40.377337 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.377313    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/38d2b3ac-d681-4628-9154-8f6daf1a711b-podres\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.377441 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.377345    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38d2b3ac-d681-4628-9154-8f6daf1a711b-lib-modules\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.377441 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.377387    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw6qs\" (UniqueName: \"kubernetes.io/projected/38d2b3ac-d681-4628-9154-8f6daf1a711b-kube-api-access-qw6qs\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.377528 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.377437    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/38d2b3ac-d681-4628-9154-8f6daf1a711b-proc\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.377528 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.377468    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38d2b3ac-d681-4628-9154-8f6daf1a711b-sys\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.478824 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.478792    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/38d2b3ac-d681-4628-9154-8f6daf1a711b-podres\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.478930 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.478834    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38d2b3ac-d681-4628-9154-8f6daf1a711b-lib-modules\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.478930 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.478877    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw6qs\" (UniqueName: \"kubernetes.io/projected/38d2b3ac-d681-4628-9154-8f6daf1a711b-kube-api-access-qw6qs\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.478930 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.478925    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/38d2b3ac-d681-4628-9154-8f6daf1a711b-proc\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.479096 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.478948    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38d2b3ac-d681-4628-9154-8f6daf1a711b-sys\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.479096 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.478947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/38d2b3ac-d681-4628-9154-8f6daf1a711b-podres\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.479096 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.479013    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38d2b3ac-d681-4628-9154-8f6daf1a711b-lib-modules\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.479096 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.479023    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38d2b3ac-d681-4628-9154-8f6daf1a711b-sys\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.479096 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.479053    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/38d2b3ac-d681-4628-9154-8f6daf1a711b-proc\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.486545 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.486518    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw6qs\" (UniqueName: \"kubernetes.io/projected/38d2b3ac-d681-4628-9154-8f6daf1a711b-kube-api-access-qw6qs\") pod \"perf-node-gather-daemonset-jppkj\" (UID: \"38d2b3ac-d681-4628-9154-8f6daf1a711b\") " pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.568102 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.568048    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:40.696583 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:40.696541    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"]
Jul 28 21:06:40.698301 ip-10-0-133-72 kubenswrapper[2558]: W0728 21:06:40.698268    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod38d2b3ac_d681_4628_9154_8f6daf1a711b.slice/crio-3e105524d4769d107af189e57d32b1c2811a229fabc67e73970e4c3edc9775f3 WatchSource:0}: Error finding container 3e105524d4769d107af189e57d32b1c2811a229fabc67e73970e4c3edc9775f3: Status 404 returned error can't find the container with id 3e105524d4769d107af189e57d32b1c2811a229fabc67e73970e4c3edc9775f3
Jul 28 21:06:41.214010 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:41.213979    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj" event={"ID":"38d2b3ac-d681-4628-9154-8f6daf1a711b","Type":"ContainerStarted","Data":"d911000e6045105e0647e4742c11c9fcaf9e2179567f8126338efd6c486be371"}
Jul 28 21:06:41.214193 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:41.214034    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj" event={"ID":"38d2b3ac-d681-4628-9154-8f6daf1a711b","Type":"ContainerStarted","Data":"3e105524d4769d107af189e57d32b1c2811a229fabc67e73970e4c3edc9775f3"}
Jul 28 21:06:41.214193 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:41.214057    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:41.231117 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:41.231075    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj" podStartSLOduration=1.231059905 podStartE2EDuration="1.231059905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 21:06:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:06:41.230249487 +0000 UTC m=+3478.472342710" watchObservedRunningTime="2026-07-28 21:06:41.231059905 +0000 UTC m=+3478.473153128"
Jul 28 21:06:41.554974 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:41.554946    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-74dbfd7fbd-mg6k6_80390a5f-7218-44c8-b025-6d2c0c6d542a/console/0.log"
Jul 28 21:06:43.243074 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:43.243044    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zl2gn_73595785-7c8d-48c2-9d25-8b2af9520034/dns/0.log"
Jul 28 21:06:43.265344 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:43.265320    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zl2gn_73595785-7c8d-48c2-9d25-8b2af9520034/kube-rbac-proxy/0.log"
Jul 28 21:06:43.287423 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:43.287400    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7hg4x_98a9c952-fdae-4bdd-9944-f68a30c41f83/dns-node-resolver/0.log"
Jul 28 21:06:43.651992 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:43.651963    2558 scope.go:117] "RemoveContainer" containerID="15a31fa1b3c4b0ab3e85168ae586a203d2cbf176865a76340c23aff4226e829c"
Jul 28 21:06:43.659952 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:43.659933    2558 scope.go:117] "RemoveContainer" containerID="78bd7e2e983d3bfe27a531fe1291ec6095b5f44dcfcbf49866410cb172c17b31"
Jul 28 21:06:43.948968 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:43.948896    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z4zjn_1545e656-1004-4d5a-aee1-7bf87e8017b2/node-ca/0.log"
Jul 28 21:06:45.147987 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:45.147957    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bb564c89-wtzwv_115f81c7-6e64-414a-aa77-fac6a2dc2224/kube-auth-proxy/0.log"
Jul 28 21:06:46.146068 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:46.146039    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nk69s_c9b4e7cf-6cbf-4067-9491-f5c3b83663a0/serve-healthcheck-canary/0.log"
Jul 28 21:06:46.699680 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:46.699653    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lsbx7_4a7a6337-4154-40f0-bc94-f66cd5da615b/kube-rbac-proxy/0.log"
Jul 28 21:06:46.718408 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:46.718384    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lsbx7_4a7a6337-4154-40f0-bc94-f66cd5da615b/exporter/0.log"
Jul 28 21:06:46.736853 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:46.736834    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lsbx7_4a7a6337-4154-40f0-bc94-f66cd5da615b/extractor/0.log"
Jul 28 21:06:47.229755 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:47.229729    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cm86f/perf-node-gather-daemonset-jppkj"
Jul 28 21:06:49.463970 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:49.463939    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74ff7fbd5d-kbj97_18c742c1-226f-4208-a374-e7cbec444d18/manager/0.log"
Jul 28 21:06:49.484202 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:49.484180    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74ff7fbd5d-kbj97_18c742c1-226f-4208-a374-e7cbec444d18/copy-manifests/0.log"
Jul 28 21:06:51.003327 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:51.003298    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6f8978bd54-bnj44_00d9d98b-0bac-4480-ba10-6361452a37a8/manager/0.log"
Jul 28 21:06:51.055514 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:51.055455    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-9r25q_74f3fecf-92b1-4a04-ae8d-c71ccdd8f973/openshift-lws-operator/0.log"
Jul 28 21:06:51.087314 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:51.087295    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bb564c89-wtzwv_115f81c7-6e64-414a-aa77-fac6a2dc2224/kube-auth-proxy/0.log"
Jul 28 21:06:52.909932 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:52.909908    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bb564c89-wtzwv_115f81c7-6e64-414a-aa77-fac6a2dc2224/kube-auth-proxy/0.log"
Jul 28 21:06:57.691811 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:57.691731    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrgg9_173e52b7-2fc0-4c1a-8036-96d743ff8dd1/kube-multus-additional-cni-plugins/0.log"
Jul 28 21:06:57.710808 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:57.710783    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrgg9_173e52b7-2fc0-4c1a-8036-96d743ff8dd1/egress-router-binary-copy/0.log"
Jul 28 21:06:57.729098 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:57.729061    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrgg9_173e52b7-2fc0-4c1a-8036-96d743ff8dd1/cni-plugins/0.log"
Jul 28 21:06:57.751864 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:57.751847    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrgg9_173e52b7-2fc0-4c1a-8036-96d743ff8dd1/bond-cni-plugin/0.log"
Jul 28 21:06:57.771050 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:57.771031    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrgg9_173e52b7-2fc0-4c1a-8036-96d743ff8dd1/routeoverride-cni/0.log"
Jul 28 21:06:57.791071 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:57.791050    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrgg9_173e52b7-2fc0-4c1a-8036-96d743ff8dd1/whereabouts-cni-bincopy/0.log"
Jul 28 21:06:57.809368 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:57.809349    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrgg9_173e52b7-2fc0-4c1a-8036-96d743ff8dd1/whereabouts-cni/0.log"
Jul 28 21:06:57.841082 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:57.841057    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-b72f2_b2678ad9-8742-49c3-98f7-7c22b69edb12/kube-multus/0.log"
Jul 28 21:06:57.993595 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:57.993504    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pghh9_232db419-1a78-4e78-a6fe-da6c6bc7e72f/network-metrics-daemon/0.log"
Jul 28 21:06:58.010770 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:58.010744    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pghh9_232db419-1a78-4e78-a6fe-da6c6bc7e72f/kube-rbac-proxy/0.log"
Jul 28 21:06:58.860099 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:58.860071    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-controller/0.log"
Jul 28 21:06:58.876950 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:58.876926    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/1.log"
Jul 28 21:06:58.877087 ip-10-0-133-72 kubenswrapper[2558]: I0728 21:06:58.877000    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-czcmx_85d75f9d-263e-430c-9493-429d83d28190/ovn-acl-logging/0.log"