Jul 28 22:02:11.421277 ip-10-0-128-9 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 22:02:11.421291 ip-10-0-128-9 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 22:02:11.421300 ip-10-0-128-9 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 22:02:11.421629 ip-10-0-128-9 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 22:02:21.605230 ip-10-0-128-9 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 22:02:21.605254 ip-10-0-128-9 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 62aa23ecef0c43bdb7c58a45651bdf6c --
Jul 28 22:04:15.892092 ip-10-0-128-9 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 22:04:16.226530 ip-10-0-128-9 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 22:04:16.226530 ip-10-0-128-9 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 22:04:16.226530 ip-10-0-128-9 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 22:04:16.226530 ip-10-0-128-9 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 22:04:16.226530 ip-10-0-128-9 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 22:04:16.228611 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.228526    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 22:04:16.233267 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233252    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:04:16.233267 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233266    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233270    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233274    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233277    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233280    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233282    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233285    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233288    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233291    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233294    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233297    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233300    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233302    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233305    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233307    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233311    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233313    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233316    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233319    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233322    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:04:16.233330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233324    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233327    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233330    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233333    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233335    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233338    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233343    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233347    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233350    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233352    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233355    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233358    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233360    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233363    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233365    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233368    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233370    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233373    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233378    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:04:16.233848 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233381    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233385    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233388    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233391    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233394    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233396    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233399    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233401    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233404    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233407    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233409    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233411    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233414    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233416    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233419    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233422    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233425    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233428    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233430    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233433    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:04:16.234298 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233435    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233438    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233441    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233443    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233446    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233448    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233450    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233453    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233455    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233458    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233460    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233463    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233466    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233469    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233471    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233474    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233477    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233480    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233482    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:04:16.234816 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233485    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233487    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233490    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233493    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233496    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233498    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233501    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233504    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233506    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233509    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233511    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233514    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233516    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233519    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233521    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233524    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233526    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233529    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233531    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.233534    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:04:16.235297 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234236    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234242    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234245    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234248    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234251    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234254    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234257    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234260    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234263    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234266    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234268    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234271    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234274    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234277    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234279    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234282    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234286    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234290    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234293    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234297    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:04:16.235831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234301    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234304    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234308    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234312    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234315    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234318    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234321    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234324    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234327    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234329    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234332    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234335    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234337    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234340    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234342    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234344    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234347    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234349    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234353    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:04:16.236330 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234356    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234359    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234361    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234365    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234369    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234372    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234374    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234377    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234379    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234382    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234384    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234387    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234389    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234392    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234394    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234397    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234399    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234402    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234404    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234407    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:04:16.236832 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234409    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234412    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234414    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234416    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234419    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234421    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234424    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234426    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234429    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234431    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234434    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234436    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234439    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234442    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234445    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234447    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234450    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234452    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234454    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234457    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:04:16.237318 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234459    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234462    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234464    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234467    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234469    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234472    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234475    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234477    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234479    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234482    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234485    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234487    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234490    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234492    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234494    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234497    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234499    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234502    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234504    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234507    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:04:16.237789 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234874    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234879    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234882    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234885    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234888    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234891    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234895    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234899    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234902    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234905    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234907    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234910    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234913    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234915    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234919    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234921    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234924    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234926    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234929    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234931    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:04:16.238278 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234934    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234936    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234939    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234941    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234944    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234946    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234949    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234952    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234954    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234957    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234960    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234962    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234965    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234969    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234973    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234975    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234978    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234981    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234984    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234987    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:04:16.238767 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234989    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234992    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234995    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.234997    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235002    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235005    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235008    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235012    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235015    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235017    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235020    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235024    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235028    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235031    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235034    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235037    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235040    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235043    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235045    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:04:16.239265 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235048    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235051    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235053    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235056    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235058    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235061    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235068    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235071    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235073    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235076    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235079    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235081    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235084    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235087    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235089    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235092    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235095    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235098    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235100    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235103    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:04:16.239745 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235105    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235108    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235110    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235113    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235115    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235118    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235120    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235123    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235125    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235128    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235130    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235133    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235135    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235138    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235140    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235143    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235145    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235150    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235154    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235157    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235227    2583 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 22:04:16.240238 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235233    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235240    2583 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235244    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235249    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235252    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235256    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235260    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235263    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235266    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235269    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235273    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235276    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235278    2583 flags.go:64] FLAG: --cgroup-root=""
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235281    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235285    2583 flags.go:64] FLAG: --client-ca-file=""
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235288    2583 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235290    2583 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235295    2583 flags.go:64] FLAG: --cluster-domain=""
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235298    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235301    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235304    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235308    2583 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235312    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 22:04:16.240740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235315    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235318    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235322    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235325    2583 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235327    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235332    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235336    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235340    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235344    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235347    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235350    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235353    2583 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235356    2583 flags.go:64] FLAG: --enable-server="true"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235359    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235364    2583 flags.go:64] FLAG: --event-burst="100"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235367    2583 flags.go:64] FLAG: --event-qps="50"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235370    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235373    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235376    2583 flags.go:64] FLAG: --eviction-hard=""
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235380    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235383    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235386    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235389    2583 flags.go:64] FLAG: --eviction-soft=""
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235392    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235395    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 22:04:16.241403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235398    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235401    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235404    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235407    2583 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235410    2583 flags.go:64] FLAG: --feature-gates=""
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235414    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235417    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235420    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235423    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235426    2583 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235429    2583 flags.go:64] FLAG: --help="false"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235432    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235436    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235440    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235443    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235447    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235450    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235453    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235460    2583 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235463    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235466    2583 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235469    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235473    2583 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235476    2583 flags.go:64] FLAG: --kube-reserved=""
Jul 28 22:04:16.242214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235479    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235481    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235485    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235487    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235490    2583 flags.go:64] FLAG: --lock-file=""
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235493    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235496    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235499    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235504    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235507    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235510    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235513    2583 flags.go:64] FLAG: --logging-format="text"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235516    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235519    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235522    2583 flags.go:64] FLAG: --manifest-url=""
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235525    2583 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235529    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235532    2583 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235536    2583 flags.go:64] FLAG: --max-pods="110"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235539    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235543    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235547    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235550    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235553    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235556    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 22:04:16.243232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235559    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235566    2583 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235569    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235572    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235575    2583 flags.go:64] FLAG: --pod-cidr=""
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235579    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235585    2583 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235587    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235590    2583 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235593    2583 flags.go:64] FLAG: --port="10250"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235597    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235600    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08ecfca8af6328d5c"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235603    2583 flags.go:64] FLAG: --qos-reserved=""
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235606    2583 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235608    2583 flags.go:64] FLAG: --register-node="true"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235611    2583 flags.go:64] FLAG: --register-with-taints=""
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235615    2583 flags.go:64] FLAG: --registry-burst="10"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235618    2583 flags.go:64] FLAG: --registry-qps="5"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235621    2583 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235624    2583 flags.go:64] FLAG: --reserved-memory=""
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235627    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235630    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235633    2583 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235636    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235639    2583 flags.go:64] FLAG: --runonce="false"
Jul 28 22:04:16.244179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235642    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235645    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235648    2583 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235653    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235656    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235659    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235662    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235665    2583 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235668    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235671    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235674    2583 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235677    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235680    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235683    2583 flags.go:64] FLAG: --system-cgroups=""
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235686    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235691    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235694    2583 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235697    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235701    2583 flags.go:64] FLAG: --tls-min-version=""
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235704    2583 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235707    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235709    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235712    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235724    2583 flags.go:64] FLAG: --v="2"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235728    2583 flags.go:64] FLAG: --version="false"
Jul 28 22:04:16.244968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235732    2583 flags.go:64] FLAG: --vmodule=""
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235735    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.235739    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235870    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235874    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235877    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235880    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235883    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235885    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235888    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235893    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235896    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235898    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235901    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235903    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235906    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235909    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235911    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235914    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235916    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:04:16.246036 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235919    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235923    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235927    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235930    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235932    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235935    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235937    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235940    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235942    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235945    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235948    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235950    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235954    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235958    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235961    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235965    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235967    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235970    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235973    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:04:16.246563 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235976    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235979    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235981    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235986    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235989    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235991    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235994    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235997    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.235999    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236002    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236005    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236007    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236010    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236013    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236015    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236018    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236021    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236023    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236026    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236029    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:04:16.247362 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236031    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236034    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236037    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236039    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236042    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236045    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236047    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236050    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236053    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236055    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236058    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236060    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236063    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236066    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236069    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236073    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236076    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236079    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236081    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236084    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:04:16.248186 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236086    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236089    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236091    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236094    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236098    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236101    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236104    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236107    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236111    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236113    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236116    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236119    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236122    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236125    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236127    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236130    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236133    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236135    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236138    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:04:16.248934 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236141    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236145    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236147    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.236150    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.236683    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.243939    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.243958    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.244601    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.244614    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244701    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244709    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244714    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244720    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244724    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:04:16.249477 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244729    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244734    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244739    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244744    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244748    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244752    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244756    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244761    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244766    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244770    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244775    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244779    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244784    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244788    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244808    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244812    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244816    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244820    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244824    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244828    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:04:16.249847 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244832    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244836    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244840    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244845    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244851    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244855    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244860    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244864    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244868    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244873    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244877    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244882    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244886    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244890    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244894    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244898    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244903    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244907    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244911    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244916    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:04:16.250387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244920    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244924    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244929    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244933    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244937    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244942    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244946    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244951    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244955    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244959    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244963    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244967    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244971    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244977    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244982    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244988    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244993    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.244998    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245002    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:04:16.250950 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245007    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245012    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245016    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245021    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245025    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245029    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245033    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245039    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245047    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245054    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245058    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245063    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245068    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245073    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245077    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245081    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245086    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245090    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245094    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:04:16.251428 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245098    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245103    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245107    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245111    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245115    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245119    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245124    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245128    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245132    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245138    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245143    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245147    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245153    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245157    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245162    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:04:16.251989 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245166    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.245175    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245355    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245363    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245367    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245372    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245377    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245382    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245389    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245394    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245399    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245403    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245408    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245413    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:04:16.252596 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245418    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245423    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245427    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245431    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245435    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245440    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245444    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245448    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245452    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245457    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245461    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245465    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245471    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245475    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245480    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245484    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245489    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245493    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245498    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245502    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:04:16.253193 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245506    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245511    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245515    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245519    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245523    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245527    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245531    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245535    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245539    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245544    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245548    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245552    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245556    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245560    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245564    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245568    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245572    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245576    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245581    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245585    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:04:16.254001 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245589    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245593    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245597    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245601    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245606    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245610    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245614    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245619    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245623    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245628    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245632    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245636    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245640    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245644    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245648    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245653    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245657    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245662    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245667    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245671    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:04:16.254751 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245675    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245679    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245683    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245690    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245695    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245700    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245704    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245708    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245713    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245717    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245722    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245726    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245730    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245734    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245738    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245742    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245747    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245751    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245755    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:04:16.255396 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245759    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245763    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245768    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245773    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245777    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245782    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245786    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:16.245790    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.245814    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.245957    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.248620    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.249951    2583 server.go:1019] "Starting client certificate rotation"
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.250050    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.250096    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 22:04:16.255967 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.252473    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 22:04:16.257060 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.257043    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 22:04:16.265542 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.265525    2583 log.go:25] "Validated CRI v1 runtime API"
Jul 28 22:04:16.268172 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.268159    2583 log.go:25] "Validated CRI v1 image API"
Jul 28 22:04:16.270124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.270109    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 22:04:16.272339 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.272317    2583 fs.go:135] Filesystem UUIDs: map[619b4fcc-a851-48ae-88cd-3bcf43628794:/dev/nvme0n1p3 6c595d70-9d3d-4185-b7fb-27433b9de8df:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 28 22:04:16.272414 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.272339    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 22:04:16.277665 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.277546    2583 manager.go:216] Machine: {Timestamp:2026-07-28 22:04:16.276263892 +0000 UTC m=+0.284129469 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103110 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26b2232ec2d05ef58ab7d051fd0343 SystemUUID:ec26b223-2ec2-d05e-f58a-b7d051fd0343 BootID:62aa23ec-ef0c-43bd-b7c5-8a45651bdf6c Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3b:81:70:1e:d9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3b:81:70:1e:d9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:49:ac:34:69:8c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 22:04:16.277665 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.277657    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 22:04:16.277789 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.277728    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 22:04:16.278544 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.278528    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 22:04:16.280312 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.280284    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 22:04:16.280452 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.280315    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-9.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 22:04:16.281043 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.281033    2583 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 22:04:16.281074 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.281046    2583 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 22:04:16.281074 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.281054    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 22:04:16.281636 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.281626    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 22:04:16.282588 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.282578    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 22:04:16.285333 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.285322    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 22:04:16.285469 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.285460    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 22:04:16.285506 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.285488    2583 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 22:04:16.285506 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.285499    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 22:04:16.285562 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.285513    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 22:04:16.285562 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.285524    2583 kubelet.go:398] "Adding apiserver pod source"
Jul 28 22:04:16.285562 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.285533    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 22:04:16.285917 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.285903    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 22:04:16.285956 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.285928    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 22:04:16.287758 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.287712    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 22:04:16.289742 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.289723    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 22:04:16.289813 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.289758    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 22:04:16.291355 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.291344    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 22:04:16.291404 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.291361    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 22:04:16.291404 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.291367    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 22:04:16.291404 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.291373    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 22:04:16.291404 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.291381    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 22:04:16.291404 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.291387    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 22:04:16.291404 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.291395    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 22:04:16.291404 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.291400    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 22:04:16.291589 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.291413    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 22:04:16.291589 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.291418    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 22:04:16.291589 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.291427    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 22:04:16.291589 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.291432    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 22:04:16.292151 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.292141    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 22:04:16.292151 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.292150    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 22:04:16.295402 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.295390    2583 server.go:1268] "Started kubelet"
Jul 28 22:04:16.295615 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.295551    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 22:04:16.295733 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.295629    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 22:04:16.296238 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.296160    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 22:04:16.296378 ip-10-0-128-9 systemd[1]: Started Kubernetes Kubelet.
Jul 28 22:04:16.298906 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.298874    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 22:04:16.300922 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.300898    2583 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 22:04:16.301318 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.301285    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 22:04:16.303290 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.303272    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-9.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 22:04:16.303866 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.303843    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-9.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 22:04:16.303917 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.303844    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 22:04:16.304074 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.303315    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-9.ec2.internal.18c6934bda6ca7d3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-9.ec2.internal,UID:ip-10-0-128-9.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-9.ec2.internal,},FirstTimestamp:2026-07-28 22:04:16.295364563 +0000 UTC m=+0.303230112,LastTimestamp:2026-07-28 22:04:16.295364563 +0000 UTC m=+0.303230112,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-9.ec2.internal,}"
Jul 28 22:04:16.306825 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.306807    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 22:04:16.306923 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.306826    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 22:04:16.306923 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.306909    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:16.307531 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.307504    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 22:04:16.307614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.307586    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 22:04:16.307614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.307555    2583 factory.go:55] Registering systemd factory
Jul 28 22:04:16.307707 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.307510    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 22:04:16.307707 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.307621    2583 factory.go:223] Registration of the systemd container factory successfully
Jul 28 22:04:16.307707 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.307699    2583 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 22:04:16.307872 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.307709    2583 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 22:04:16.307872 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.307852    2583 factory.go:153] Registering CRI-O factory
Jul 28 22:04:16.307872 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.307866    2583 factory.go:223] Registration of the crio container factory successfully
Jul 28 22:04:16.307986 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.307913    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 22:04:16.307986 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.307937    2583 factory.go:103] Registering Raw factory
Jul 28 22:04:16.307986 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.307951    2583 manager.go:1196] Started watching for new ooms in manager
Jul 28 22:04:16.308608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.308595    2583 manager.go:349] Starting recovery of all containers
Jul 28 22:04:16.314989 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.314950    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 22:04:16.318134 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.318097    2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-9.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 22:04:16.318217 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.318186    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 22:04:16.319828 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.319814    2583 manager.go:354] Recovery completed
Jul 28 22:04:16.321094 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.321075    2583 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 28 22:04:16.324152 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.324138    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:16.324618 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.324601    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sxbkz"
Jul 28 22:04:16.326017 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.326001    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:16.326105 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.326037    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:16.326105 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.326047    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:16.326589 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.326571    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 22:04:16.326670 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.326590    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 22:04:16.326670 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.326605    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:16.326670 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.326609    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 22:04:16.327304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.327288    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:16.327362 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.327319    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:16.327362 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.327333    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:16.329273 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.329261    2583 policy_none.go:49] "None policy: Start"
Jul 28 22:04:16.329273 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.329275    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 22:04:16.329361 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.329285    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 22:04:16.331209 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.331197    2583 policy_none.go:47] "Start"
Jul 28 22:04:16.332313 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.332204    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sxbkz"
Jul 28 22:04:16.334545 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.334479    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-9.ec2.internal.18c6934bdc405e0c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-9.ec2.internal,UID:ip-10-0-128-9.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-9.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-9.ec2.internal,},FirstTimestamp:2026-07-28 22:04:16.326016524 +0000 UTC m=+0.333882073,LastTimestamp:2026-07-28 22:04:16.326016524 +0000 UTC m=+0.333882073,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-9.ec2.internal,}"
Jul 28 22:04:16.365345 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.365332    2583 manager.go:337] "Starting Device Plugin manager"
Jul 28 22:04:16.365414 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.365366    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 22:04:16.365414 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.365376    2583 server.go:87] "Starting device plugin registration server"
Jul 28 22:04:16.365638 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.365623    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 22:04:16.365746 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.365637    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 22:04:16.365746 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.365734    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 22:04:16.365920 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.365842    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 22:04:16.365920 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.365852    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 22:04:16.366353 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.366330    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 22:04:16.366407 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.366389    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:16.424836 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.424813    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 22:04:16.426970 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.424860    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 22:04:16.426970 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.424883    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 22:04:16.426970 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.424920    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 22:04:16.427587 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.427572    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:04:16.465923 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.465903    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:16.466744 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.466725    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:16.466844 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.466758    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:16.466844 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.466773    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:16.466844 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.466821    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.475647 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.475632    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.475712 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.475652    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-9.ec2.internal\": node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:16.501410 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.501335    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:16.525943 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.525912    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-9.ec2.internal"]
Jul 28 22:04:16.526019 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.525975    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:16.526777 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.526762    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:16.526860 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.526789    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:16.526860 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.526820    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:16.529157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.529145    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:16.529314 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.529299    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.529365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.529333    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:16.529878 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.529862    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:16.529939 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.529890    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:16.529939 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.529862    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:16.529939 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.529923    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:16.529939 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.529935    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:16.530107 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.529900    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:16.532139 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.532126    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.532212 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.532149    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:16.532754 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.532742    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:16.532844 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.532768    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:16.532844 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.532780    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:16.560519 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.560501    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-9.ec2.internal\" not found" node="ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.564813 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.564784    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-9.ec2.internal\" not found" node="ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.601784 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.601766    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:16.608573 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.608555    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/919572f6e569012b430fa28bc75de12b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal\" (UID: \"919572f6e569012b430fa28bc75de12b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.608635 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.608577    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/41dbbb64873b28a94876546d4a4c4566-config\") pod \"kube-apiserver-proxy-ip-10-0-128-9.ec2.internal\" (UID: \"41dbbb64873b28a94876546d4a4c4566\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.608635 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.608594    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/919572f6e569012b430fa28bc75de12b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal\" (UID: \"919572f6e569012b430fa28bc75de12b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.702489 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.702463    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:16.708746 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.708725    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/919572f6e569012b430fa28bc75de12b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal\" (UID: \"919572f6e569012b430fa28bc75de12b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.708818 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.708754    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/919572f6e569012b430fa28bc75de12b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal\" (UID: \"919572f6e569012b430fa28bc75de12b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.708818 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.708771    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/41dbbb64873b28a94876546d4a4c4566-config\") pod \"kube-apiserver-proxy-ip-10-0-128-9.ec2.internal\" (UID: \"41dbbb64873b28a94876546d4a4c4566\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.708944 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.708837    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/41dbbb64873b28a94876546d4a4c4566-config\") pod \"kube-apiserver-proxy-ip-10-0-128-9.ec2.internal\" (UID: \"41dbbb64873b28a94876546d4a4c4566\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.708944 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.708835    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/919572f6e569012b430fa28bc75de12b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal\" (UID: \"919572f6e569012b430fa28bc75de12b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.708944 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.708900    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/919572f6e569012b430fa28bc75de12b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal\" (UID: \"919572f6e569012b430fa28bc75de12b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.803324 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.803249    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:16.866467 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.866435    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.868764 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:16.868745    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:16.903582 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:16.903558    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:17.009577 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:17.009546    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:17.114868 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:17.114790    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:17.215272 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:17.215233    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:17.250726 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.250685    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 22:04:17.251195 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.250889    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 22:04:17.315932 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:17.315905    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:17.338069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.338028    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 21:59:17 +0000 UTC" deadline="2028-02-29 21:08:43.285930088 +0000 UTC"
Jul 28 22:04:17.338069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.338066    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13943h4m25.947867071s"
Jul 28 22:04:17.385927 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.385869    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:04:17.386759 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.386740    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:04:17.386849 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.386777    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:04:17.386849 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.386787    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:04:17.416507 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:17.416480    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:17.470369 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.470345    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:04:17.517256 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:17.517229    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:17.618107 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:17.618074    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:17.655907 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:17.655877    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41dbbb64873b28a94876546d4a4c4566.slice/crio-5552cde37c6306237ef44d817f5eb79118aa3a5008e3f38e7b4a245d92612496 WatchSource:0}: Error finding container 5552cde37c6306237ef44d817f5eb79118aa3a5008e3f38e7b4a245d92612496: Status 404 returned error can't find the container with id 5552cde37c6306237ef44d817f5eb79118aa3a5008e3f38e7b4a245d92612496
Jul 28 22:04:17.661952 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.661936    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:04:17.667581 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:17.667558    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod919572f6e569012b430fa28bc75de12b.slice/crio-b75ce49ad0568d6699e734b17f093f6ab1a83f87c06a9b270ab291f5ea238a0d WatchSource:0}: Error finding container b75ce49ad0568d6699e734b17f093f6ab1a83f87c06a9b270ab291f5ea238a0d: Status 404 returned error can't find the container with id b75ce49ad0568d6699e734b17f093f6ab1a83f87c06a9b270ab291f5ea238a0d
Jul 28 22:04:17.718881 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:17.718854    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-9.ec2.internal\" not found"
Jul 28 22:04:17.727132 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.727117    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:04:17.804323 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.804299    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:04:17.807196 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.807181    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:17.824783 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.824762    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 22:04:17.825632 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.825618    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-9.ec2.internal"
Jul 28 22:04:17.832937 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:17.832920    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 22:04:18.287069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.287035    2583 apiserver.go:52] "Watching apiserver"
Jul 28 22:04:18.293532 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.293504    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 22:04:18.296123 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.296094    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-v66pk","openshift-ovn-kubernetes/ovnkube-node-hctzl","kube-system/global-pull-secret-syncer-t5bf9","kube-system/konnectivity-agent-4hv59","kube-system/kube-apiserver-proxy-ip-10-0-128-9.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2","openshift-cluster-node-tuning-operator/tuned-mgxvb","openshift-multus/multus-additional-cni-plugins-zjwtm","openshift-multus/network-metrics-daemon-5vh87","openshift-dns/node-resolver-lkmdh","openshift-image-registry/node-ca-zvgg7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal","openshift-multus/multus-wqv74","openshift-network-diagnostics/network-check-target-btcz4"]
Jul 28 22:04:18.298908 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.298881    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.301167 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.301142    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 22:04:18.301272 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.301194    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wdbc4\""
Jul 28 22:04:18.301388 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.301371    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 22:04:18.301454 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.301399    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zvgg7"
Jul 28 22:04:18.303516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.303366    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 22:04:18.303516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.303392    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 22:04:18.303516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.303413    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-95vz5\""
Jul 28 22:04:18.303714 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.303656    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 22:04:18.303714 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.303669    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:18.303817 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:18.303758    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:18.305908 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.305890    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4hv59"
Jul 28 22:04:18.307251 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.307229    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 22:04:18.307654 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.307636    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 22:04:18.308047 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.308028    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8c8hf\""
Jul 28 22:04:18.308444 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.308288    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 22:04:18.308836 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.308786    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:18.308945 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:18.308892    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:18.311313 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.311070    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.313465 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.313302    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 22:04:18.313465 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.313314    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-p794c\""
Jul 28 22:04:18.313751 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.313731    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:18.313847 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:18.313807    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:18.313847 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.313820    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 22:04:18.313847 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.313827    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 22:04:18.316089 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.316066    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.316836 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.316817    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 22:04:18.318103 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.318082    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 22:04:18.318573 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.318375    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 22:04:18.318771 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.318747    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p7r8b\""
Jul 28 22:04:18.318889 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.318794    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 22:04:18.318953 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.318747    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 22:04:18.319319 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.319126    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.321263 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.321167    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lsxdk\""
Jul 28 22:04:18.321886 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.321481    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 22:04:18.321886 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.321510    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lkmdh"
Jul 28 22:04:18.321886 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.321520    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 22:04:18.321886 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.321759    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 22:04:18.321886 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.321822    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 22:04:18.322855 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.322835    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 22:04:18.322950 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.322863    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 22:04:18.322950 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.322887    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/db0604f9-fa11-4595-9276-4430c567837b-etc-tuned\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.322950 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.322919    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/63b7aaeb-cc08-452a-89b7-59cf107dce99-serviceca\") pod \"node-ca-zvgg7\" (UID: \"63b7aaeb-cc08-452a-89b7-59cf107dce99\") " pod="openshift-image-registry/node-ca-zvgg7"
Jul 28 22:04:18.323112 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.322966    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/939e5bee-11fa-4cc1-852b-49fba60700ee-konnectivity-ca\") pod \"konnectivity-agent-4hv59\" (UID: \"939e5bee-11fa-4cc1-852b-49fba60700ee\") " pod="kube-system/konnectivity-agent-4hv59"
Jul 28 22:04:18.323112 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.322993    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-kubernetes\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.323112 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323017    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-sysctl-d\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.323112 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323041    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-sysctl-conf\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.323112 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323075    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-systemd\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.323112 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323097    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db0604f9-fa11-4595-9276-4430c567837b-tmp\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.323407 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323227    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/63b7aaeb-cc08-452a-89b7-59cf107dce99-host\") pod \"node-ca-zvgg7\" (UID: \"63b7aaeb-cc08-452a-89b7-59cf107dce99\") " pod="openshift-image-registry/node-ca-zvgg7"
Jul 28 22:04:18.323407 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323265    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-modprobe-d\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.323407 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323294    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-sys\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.323407 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323330    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-host\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.323407 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323370    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pg7g\" (UniqueName: \"kubernetes.io/projected/63b7aaeb-cc08-452a-89b7-59cf107dce99-kube-api-access-7pg7g\") pod \"node-ca-zvgg7\" (UID: \"63b7aaeb-cc08-452a-89b7-59cf107dce99\") " pod="openshift-image-registry/node-ca-zvgg7"
Jul 28 22:04:18.323654 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323415    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dae75d3b-5868-4cd2-b7e4-45f6a145b875-kubelet-config\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:18.323654 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323442    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dae75d3b-5868-4cd2-b7e4-45f6a145b875-dbus\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:18.323654 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323471    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/939e5bee-11fa-4cc1-852b-49fba60700ee-agent-certs\") pod \"konnectivity-agent-4hv59\" (UID: \"939e5bee-11fa-4cc1-852b-49fba60700ee\") " pod="kube-system/konnectivity-agent-4hv59"
Jul 28 22:04:18.323654 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323495    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-sysconfig\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.323654 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323563    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-var-lib-kubelet\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.323654 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323588    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wkb6\" (UniqueName: \"kubernetes.io/projected/db0604f9-fa11-4595-9276-4430c567837b-kube-api-access-6wkb6\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.323654 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323609    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:18.324120 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323660    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk6vq\" (UniqueName: \"kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq\") pod \"network-check-target-btcz4\" (UID: \"b83ca49e-9d70-42ee-b256-2d4d995c598e\") " pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:18.324120 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323693    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-run\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.324120 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.323735    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-lib-modules\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.324315 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.324296    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v66pk"
Jul 28 22:04:18.324667 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.324643    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 22:04:18.324726 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.324645    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-499ng\""
Jul 28 22:04:18.324816 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.324797    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 22:04:18.327217 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.327194    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7wbrc\""
Jul 28 22:04:18.327484 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.327466    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 22:04:18.327604 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.327584    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.327827 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.327813    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 22:04:18.328216 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.328197    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 22:04:18.329731 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.329702    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 22:04:18.329839 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.329822    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zclqn\""
Jul 28 22:04:18.330772 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.330748    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 22:04:18.350092 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.350075    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xpvhl"
Jul 28 22:04:18.353671 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.353648    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:04:18.358852 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.358831    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xpvhl"
Jul 28 22:04:18.408714 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.408691    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 22:04:18.424148 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424120    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-sysctl-d\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.424256 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424163    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-kubelet\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.424256 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424194    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-systemd-units\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.424256 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424244    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.424411 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424282    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-host\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.424411 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424309    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-var-lib-cni-multus\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.424411 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424346    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-sysctl-d\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.424547 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424424    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-host\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.424547 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424470    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-run-netns\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.424547 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424507    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7a8f707b-f715-437a-b41e-550b40dfe6c4-env-overrides\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.424547 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424540    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/939e5bee-11fa-4cc1-852b-49fba60700ee-agent-certs\") pod \"konnectivity-agent-4hv59\" (UID: \"939e5bee-11fa-4cc1-852b-49fba60700ee\") " pod="kube-system/konnectivity-agent-4hv59"
Jul 28 22:04:18.424738 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424569    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:18.424738 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424672    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djctg\" (UniqueName: \"kubernetes.io/projected/928ed8af-4599-402e-8dc6-5a3ca176d89f-kube-api-access-djctg\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.424876 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:18.424744    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:18.424876 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424780    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-cni-bin\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.424876 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424801    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/769507e6-3089-4849-865d-283919d41c01-hosts-file\") pod \"node-resolver-lkmdh\" (UID: \"769507e6-3089-4849-865d-283919d41c01\") " pod="openshift-dns/node-resolver-lkmdh"
Jul 28 22:04:18.425030 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:18.424886    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret podName:dae75d3b-5868-4cd2-b7e4-45f6a145b875 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:18.924862533 +0000 UTC m=+2.932728087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret") pod "global-pull-secret-syncer-t5bf9" (UID: "dae75d3b-5868-4cd2-b7e4-45f6a145b875") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:18.425030 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424931    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wfc8\" (UniqueName: \"kubernetes.io/projected/4028222a-2ecd-4831-aded-68dde7adbdae-kube-api-access-8wfc8\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.425030 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424979    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk6vq\" (UniqueName: \"kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq\") pod \"network-check-target-btcz4\" (UID: \"b83ca49e-9d70-42ee-b256-2d4d995c598e\") " pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:18.425030 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.424978    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 22:04:18.425242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425017    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/db0604f9-fa11-4595-9276-4430c567837b-etc-tuned\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.425242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425079    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24a96084-4f66-4d90-9de4-291369338782-host-slash\") pod \"iptables-alerter-v66pk\" (UID: \"24a96084-4f66-4d90-9de4-291369338782\") " pod="openshift-network-operator/iptables-alerter-v66pk"
Jul 28 22:04:18.425242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-registration-dir\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.425242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425126    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/939e5bee-11fa-4cc1-852b-49fba60700ee-konnectivity-ca\") pod \"konnectivity-agent-4hv59\" (UID: \"939e5bee-11fa-4cc1-852b-49fba60700ee\") " pod="kube-system/konnectivity-agent-4hv59"
Jul 28 22:04:18.425242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-sysctl-conf\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.425242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425169    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-systemd\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.425242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425191    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/63b7aaeb-cc08-452a-89b7-59cf107dce99-host\") pod \"node-ca-zvgg7\" (UID: \"63b7aaeb-cc08-452a-89b7-59cf107dce99\") " pod="openshift-image-registry/node-ca-zvgg7"
Jul 28 22:04:18.425242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425216    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-multus-socket-dir-parent\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.425242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425238    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-hostroot\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425258    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-log-socket\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425297    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425323    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-cnibin\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425349    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425376    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-device-dir\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425390    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-systemd\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425435    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/63b7aaeb-cc08-452a-89b7-59cf107dce99-host\") pod \"node-ca-zvgg7\" (UID: \"63b7aaeb-cc08-452a-89b7-59cf107dce99\") " pod="openshift-image-registry/node-ca-zvgg7"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425322    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-sysctl-conf\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425473    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-slash\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425504    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-var-lib-openvswitch\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425532    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-etc-openvswitch\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425555    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-cni-netd\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56qgn\" (UniqueName: \"kubernetes.io/projected/769507e6-3089-4849-865d-283919d41c01-kube-api-access-56qgn\") pod \"node-resolver-lkmdh\" (UID: \"769507e6-3089-4849-865d-283919d41c01\") " pod="openshift-dns/node-resolver-lkmdh"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425640    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-socket-dir\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425668    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bztfj\" (UniqueName: \"kubernetes.io/projected/24a96084-4f66-4d90-9de4-291369338782-kube-api-access-bztfj\") pod \"iptables-alerter-v66pk\" (UID: \"24a96084-4f66-4d90-9de4-291369338782\") " pod="openshift-network-operator/iptables-alerter-v66pk"
Jul 28 22:04:18.425774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-system-cni-dir\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425740    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-kubernetes\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db0604f9-fa11-4595-9276-4430c567837b-tmp\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425825    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dae75d3b-5868-4cd2-b7e4-45f6a145b875-dbus\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425832    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-kubernetes\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425859    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-var-lib-kubelet\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425865    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/939e5bee-11fa-4cc1-852b-49fba60700ee-konnectivity-ca\") pod \"konnectivity-agent-4hv59\" (UID: \"939e5bee-11fa-4cc1-852b-49fba60700ee\") " pod="kube-system/konnectivity-agent-4hv59"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425902    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/928ed8af-4599-402e-8dc6-5a3ca176d89f-multus-daemon-config\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425929    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-run-systemd\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425954    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7a8f707b-f715-437a-b41e-550b40dfe6c4-ovnkube-config\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425973    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7a8f707b-f715-437a-b41e-550b40dfe6c4-ovnkube-script-lib\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425998    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dae75d3b-5868-4cd2-b7e4-45f6a145b875-dbus\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.425996    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-modprobe-d\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426056    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pg7g\" (UniqueName: \"kubernetes.io/projected/63b7aaeb-cc08-452a-89b7-59cf107dce99-kube-api-access-7pg7g\") pod \"node-ca-zvgg7\" (UID: \"63b7aaeb-cc08-452a-89b7-59cf107dce99\") " pod="openshift-image-registry/node-ca-zvgg7"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426106    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-modprobe-d\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426112    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-cnibin\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426140    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24a96084-4f66-4d90-9de4-291369338782-iptables-alerter-script\") pod \"iptables-alerter-v66pk\" (UID: \"24a96084-4f66-4d90-9de4-291369338782\") " pod="openshift-network-operator/iptables-alerter-v66pk"
Jul 28 22:04:18.426516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426167    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-os-release\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426188    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426214    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkmzk\" (UniqueName: \"kubernetes.io/projected/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-kube-api-access-vkmzk\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-sysconfig\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426285    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wkb6\" (UniqueName: \"kubernetes.io/projected/db0604f9-fa11-4595-9276-4430c567837b-kube-api-access-6wkb6\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426305    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-etc-sysconfig\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426307    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs\") pod \"network-metrics-daemon-5vh87\" (UID: \"33b4bdc7-694b-45e4-8c88-3cd9f134865d\") " pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426346    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/928ed8af-4599-402e-8dc6-5a3ca176d89f-cni-binary-copy\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426370    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-var-lib-cni-bin\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426390    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-etc-kubernetes\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426410    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-run-ovn\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426437    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-run\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426460    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-lib-modules\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426485    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426513    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-run-k8s-cni-cncf-io\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426536    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/769507e6-3089-4849-865d-283919d41c01-tmp-dir\") pod \"node-resolver-lkmdh\" (UID: \"769507e6-3089-4849-865d-283919d41c01\") " pod="openshift-dns/node-resolver-lkmdh"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426568    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-run\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.427389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426584    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-etc-selinux\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426616    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-sys\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426606    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-lib-modules\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426648    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dae75d3b-5868-4cd2-b7e4-45f6a145b875-kubelet-config\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426668    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-sys\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426667    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kxkm\" (UniqueName: \"kubernetes.io/projected/33b4bdc7-694b-45e4-8c88-3cd9f134865d-kube-api-access-2kxkm\") pod \"network-metrics-daemon-5vh87\" (UID: \"33b4bdc7-694b-45e4-8c88-3cd9f134865d\") " pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426712    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-run-openvswitch\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426719    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dae75d3b-5868-4cd2-b7e4-45f6a145b875-kubelet-config\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426757    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmjnx\" (UniqueName: \"kubernetes.io/projected/7a8f707b-f715-437a-b41e-550b40dfe6c4-kube-api-access-fmjnx\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426789    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-var-lib-kubelet\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426833    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-system-cni-dir\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426858    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-os-release\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426863    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db0604f9-fa11-4595-9276-4430c567837b-var-lib-kubelet\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426884    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-multus-conf-dir\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426908    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-node-log\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426934    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426962    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/63b7aaeb-cc08-452a-89b7-59cf107dce99-serviceca\") pod \"node-ca-zvgg7\" (UID: \"63b7aaeb-cc08-452a-89b7-59cf107dce99\") " pod="openshift-image-registry/node-ca-zvgg7"
Jul 28 22:04:18.428254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.426985    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-multus-cni-dir\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.429034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.427009    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-run-netns\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.429034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.427031    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-run-multus-certs\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.429034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.427062    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a8f707b-f715-437a-b41e-550b40dfe6c4-ovn-node-metrics-cert\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.429034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.427110    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-cni-binary-copy\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.429034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.427126    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-sys-fs\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.429034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.427536    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/63b7aaeb-cc08-452a-89b7-59cf107dce99-serviceca\") pod \"node-ca-zvgg7\" (UID: \"63b7aaeb-cc08-452a-89b7-59cf107dce99\") " pod="openshift-image-registry/node-ca-zvgg7"
Jul 28 22:04:18.429034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.428969    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/db0604f9-fa11-4595-9276-4430c567837b-etc-tuned\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.429382 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.429038    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/939e5bee-11fa-4cc1-852b-49fba60700ee-agent-certs\") pod \"konnectivity-agent-4hv59\" (UID: \"939e5bee-11fa-4cc1-852b-49fba60700ee\") " pod="kube-system/konnectivity-agent-4hv59"
Jul 28 22:04:18.429382 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.429188    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db0604f9-fa11-4595-9276-4430c567837b-tmp\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.429574 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.429531    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal" event={"ID":"919572f6e569012b430fa28bc75de12b","Type":"ContainerStarted","Data":"b75ce49ad0568d6699e734b17f093f6ab1a83f87c06a9b270ab291f5ea238a0d"}
Jul 28 22:04:18.431165 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.431134    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-9.ec2.internal" event={"ID":"41dbbb64873b28a94876546d4a4c4566","Type":"ContainerStarted","Data":"5552cde37c6306237ef44d817f5eb79118aa3a5008e3f38e7b4a245d92612496"}
Jul 28 22:04:18.433733 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:18.433714    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:04:18.433826 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:18.433742    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:04:18.433826 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:18.433755    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kk6vq for pod openshift-network-diagnostics/network-check-target-btcz4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:18.433945 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:18.433829    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq podName:b83ca49e-9d70-42ee-b256-2d4d995c598e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:18.933813505 +0000 UTC m=+2.941679047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kk6vq" (UniqueName: "kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq") pod "network-check-target-btcz4" (UID: "b83ca49e-9d70-42ee-b256-2d4d995c598e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:18.437052 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.437031    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pg7g\" (UniqueName: \"kubernetes.io/projected/63b7aaeb-cc08-452a-89b7-59cf107dce99-kube-api-access-7pg7g\") pod \"node-ca-zvgg7\" (UID: \"63b7aaeb-cc08-452a-89b7-59cf107dce99\") " pod="openshift-image-registry/node-ca-zvgg7"
Jul 28 22:04:18.437571 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.437551    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wkb6\" (UniqueName: \"kubernetes.io/projected/db0604f9-fa11-4595-9276-4430c567837b-kube-api-access-6wkb6\") pod \"tuned-mgxvb\" (UID: \"db0604f9-fa11-4595-9276-4430c567837b\") " pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.527447 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527403    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-cnibin\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.527614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527449    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24a96084-4f66-4d90-9de4-291369338782-iptables-alerter-script\") pod \"iptables-alerter-v66pk\" (UID: \"24a96084-4f66-4d90-9de4-291369338782\") " pod="openshift-network-operator/iptables-alerter-v66pk"
Jul 28 22:04:18.527614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527478    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-os-release\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.527614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527506    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.527614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527531    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkmzk\" (UniqueName: \"kubernetes.io/projected/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-kube-api-access-vkmzk\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.527614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527562    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs\") pod \"network-metrics-daemon-5vh87\" (UID: \"33b4bdc7-694b-45e4-8c88-3cd9f134865d\") " pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:18.527614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527581    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/928ed8af-4599-402e-8dc6-5a3ca176d89f-cni-binary-copy\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.527614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527596    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-var-lib-cni-bin\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.527614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527612    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-etc-kubernetes\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527633    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-run-ovn\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527654    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527672    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-run-k8s-cni-cncf-io\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527688    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/769507e6-3089-4849-865d-283919d41c01-tmp-dir\") pod \"node-resolver-lkmdh\" (UID: \"769507e6-3089-4849-865d-283919d41c01\") " pod="openshift-dns/node-resolver-lkmdh"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527704    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-etc-selinux\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527736    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kxkm\" (UniqueName: \"kubernetes.io/projected/33b4bdc7-694b-45e4-8c88-3cd9f134865d-kube-api-access-2kxkm\") pod \"network-metrics-daemon-5vh87\" (UID: \"33b4bdc7-694b-45e4-8c88-3cd9f134865d\") " pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527758    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-run-openvswitch\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527774    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmjnx\" (UniqueName: \"kubernetes.io/projected/7a8f707b-f715-437a-b41e-550b40dfe6c4-kube-api-access-fmjnx\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527790    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-system-cni-dir\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527815    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-os-release\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527841    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-multus-conf-dir\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527862    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-node-log\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527898    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527920    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-multus-cni-dir\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527935    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-run-netns\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527950    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-run-multus-certs\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.527983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a8f707b-f715-437a-b41e-550b40dfe6c4-ovn-node-metrics-cert\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.527988    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-cni-binary-copy\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528008    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-sys-fs\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528023    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-kubelet\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528044    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-systemd-units\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528061    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528084    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-var-lib-cni-multus\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528104    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-run-netns\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7a8f707b-f715-437a-b41e-550b40dfe6c4-env-overrides\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528148    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djctg\" (UniqueName: \"kubernetes.io/projected/928ed8af-4599-402e-8dc6-5a3ca176d89f-kube-api-access-djctg\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528172    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-cni-bin\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528206    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/769507e6-3089-4849-865d-283919d41c01-hosts-file\") pod \"node-resolver-lkmdh\" (UID: \"769507e6-3089-4849-865d-283919d41c01\") " pod="openshift-dns/node-resolver-lkmdh"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528232    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wfc8\" (UniqueName: \"kubernetes.io/projected/4028222a-2ecd-4831-aded-68dde7adbdae-kube-api-access-8wfc8\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528257    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24a96084-4f66-4d90-9de4-291369338782-host-slash\") pod \"iptables-alerter-v66pk\" (UID: \"24a96084-4f66-4d90-9de4-291369338782\") " pod="openshift-network-operator/iptables-alerter-v66pk"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528274    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-registration-dir\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528294    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-multus-socket-dir-parent\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528310    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-hostroot\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.528595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528324    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-log-socket\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528341    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528357    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-cnibin\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528375    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528392    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-device-dir\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528409    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-slash\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528427    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-var-lib-openvswitch\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528444    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-etc-openvswitch\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528461    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-cni-netd\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528489    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56qgn\" (UniqueName: \"kubernetes.io/projected/769507e6-3089-4849-865d-283919d41c01-kube-api-access-56qgn\") pod \"node-resolver-lkmdh\" (UID: \"769507e6-3089-4849-865d-283919d41c01\") " pod="openshift-dns/node-resolver-lkmdh"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528494    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-run-netns\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528526    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-socket-dir\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528560    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bztfj\" (UniqueName: \"kubernetes.io/projected/24a96084-4f66-4d90-9de4-291369338782-kube-api-access-bztfj\") pod \"iptables-alerter-v66pk\" (UID: \"24a96084-4f66-4d90-9de4-291369338782\") " pod="openshift-network-operator/iptables-alerter-v66pk"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528562    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/769507e6-3089-4849-865d-283919d41c01-hosts-file\") pod \"node-resolver-lkmdh\" (UID: \"769507e6-3089-4849-865d-283919d41c01\") " pod="openshift-dns/node-resolver-lkmdh"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528592    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-system-cni-dir\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528644    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-var-lib-kubelet\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528669    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/928ed8af-4599-402e-8dc6-5a3ca176d89f-multus-daemon-config\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.529507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528985    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-system-cni-dir\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528975    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/769507e6-3089-4849-865d-283919d41c01-tmp-dir\") pod \"node-resolver-lkmdh\" (UID: \"769507e6-3089-4849-865d-283919d41c01\") " pod="openshift-dns/node-resolver-lkmdh"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529026    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-run-systemd\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529115    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7a8f707b-f715-437a-b41e-550b40dfe6c4-ovnkube-config\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529131    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-etc-selinux\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7a8f707b-f715-437a-b41e-550b40dfe6c4-ovnkube-script-lib\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529239    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24a96084-4f66-4d90-9de4-291369338782-iptables-alerter-script\") pod \"iptables-alerter-v66pk\" (UID: \"24a96084-4f66-4d90-9de4-291369338782\") " pod="openshift-network-operator/iptables-alerter-v66pk"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529286    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529321    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-os-release\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529328    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/928ed8af-4599-402e-8dc6-5a3ca176d89f-multus-daemon-config\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529362    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-var-lib-cni-multus\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529364    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-multus-conf-dir\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529403    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-node-log\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529416    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-run-netns\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529454    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/928ed8af-4599-402e-8dc6-5a3ca176d89f-cni-binary-copy\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529482    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-cni-bin\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529538    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529541    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-os-release\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.530365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529585    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-var-lib-cni-bin\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529630    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-etc-kubernetes\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:18.529630    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529668    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24a96084-4f66-4d90-9de4-291369338782-host-slash\") pod \"iptables-alerter-v66pk\" (UID: \"24a96084-4f66-4d90-9de4-291369338782\") " pod="openshift-network-operator/iptables-alerter-v66pk"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:18.529712    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs podName:33b4bdc7-694b-45e4-8c88-3cd9f134865d nodeName:}" failed. No retries permitted until 2026-07-28 22:04:19.029691862 +0000 UTC m=+3.037557413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs") pod "network-metrics-daemon-5vh87" (UID: "33b4bdc7-694b-45e4-8c88-3cd9f134865d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529727    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529734    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-registration-dir\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529755    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-run-ovn\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529779    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-cnibin\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529787    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-multus-socket-dir-parent\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529797    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7a8f707b-f715-437a-b41e-550b40dfe6c4-ovnkube-script-lib\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529842    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-device-dir\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529844    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529841    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529867    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-run-k8s-cni-cncf-io\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529907    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-cni-netd\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529922    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-run-systemd\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.531309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529930    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529967    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-run-openvswitch\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529969    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-log-socket\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529992    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-run-multus-certs\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.530005    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-cnibin\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.530004    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-var-lib-openvswitch\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.530043    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-system-cni-dir\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.530069    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-hostroot\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.529673    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-etc-openvswitch\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.528559    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-systemd-units\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.530112    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-host-var-lib-kubelet\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.530152    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-kubelet\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.530207    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-socket-dir\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.530220    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4028222a-2ecd-4831-aded-68dde7adbdae-sys-fs\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.530274    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a8f707b-f715-437a-b41e-550b40dfe6c4-host-slash\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.530293    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/928ed8af-4599-402e-8dc6-5a3ca176d89f-multus-cni-dir\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.530454    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-cni-binary-copy\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.530478    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7a8f707b-f715-437a-b41e-550b40dfe6c4-ovnkube-config\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.532213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.531113    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7a8f707b-f715-437a-b41e-550b40dfe6c4-env-overrides\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.533163 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.531550    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a8f707b-f715-437a-b41e-550b40dfe6c4-ovn-node-metrics-cert\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.539588 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.539524    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kxkm\" (UniqueName: \"kubernetes.io/projected/33b4bdc7-694b-45e4-8c88-3cd9f134865d-kube-api-access-2kxkm\") pod \"network-metrics-daemon-5vh87\" (UID: \"33b4bdc7-694b-45e4-8c88-3cd9f134865d\") " pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:18.540555 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.540532    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djctg\" (UniqueName: \"kubernetes.io/projected/928ed8af-4599-402e-8dc6-5a3ca176d89f-kube-api-access-djctg\") pod \"multus-wqv74\" (UID: \"928ed8af-4599-402e-8dc6-5a3ca176d89f\") " pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.541378 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.541148    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkmzk\" (UniqueName: \"kubernetes.io/projected/2a85189c-c2ee-4a08-bb3e-7ad54914bc63-kube-api-access-vkmzk\") pod \"multus-additional-cni-plugins-zjwtm\" (UID: \"2a85189c-c2ee-4a08-bb3e-7ad54914bc63\") " pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.541378 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.541334    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wfc8\" (UniqueName: \"kubernetes.io/projected/4028222a-2ecd-4831-aded-68dde7adbdae-kube-api-access-8wfc8\") pod \"aws-ebs-csi-driver-node-khkk2\" (UID: \"4028222a-2ecd-4831-aded-68dde7adbdae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.542570 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.542545    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmjnx\" (UniqueName: \"kubernetes.io/projected/7a8f707b-f715-437a-b41e-550b40dfe6c4-kube-api-access-fmjnx\") pod \"ovnkube-node-hctzl\" (UID: \"7a8f707b-f715-437a-b41e-550b40dfe6c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.542995 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.542962    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56qgn\" (UniqueName: \"kubernetes.io/projected/769507e6-3089-4849-865d-283919d41c01-kube-api-access-56qgn\") pod \"node-resolver-lkmdh\" (UID: \"769507e6-3089-4849-865d-283919d41c01\") " pod="openshift-dns/node-resolver-lkmdh"
Jul 28 22:04:18.543110 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.543088    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bztfj\" (UniqueName: \"kubernetes.io/projected/24a96084-4f66-4d90-9de4-291369338782-kube-api-access-bztfj\") pod \"iptables-alerter-v66pk\" (UID: \"24a96084-4f66-4d90-9de4-291369338782\") " pod="openshift-network-operator/iptables-alerter-v66pk"
Jul 28 22:04:18.611065 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.611034    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mgxvb"
Jul 28 22:04:18.619048 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.619016    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zvgg7"
Jul 28 22:04:18.631176 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.631153    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4hv59"
Jul 28 22:04:18.638166 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.638146    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:18.645052 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.645029    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2"
Jul 28 22:04:18.652059 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.652034    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wqv74"
Jul 28 22:04:18.659024 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.658999    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lkmdh"
Jul 28 22:04:18.667255 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.667227    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v66pk"
Jul 28 22:04:18.671476 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.671459    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zjwtm"
Jul 28 22:04:18.932272 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:18.932181    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:18.932427 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:18.932358    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:18.932427 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:18.932423    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret podName:dae75d3b-5868-4cd2-b7e4-45f6a145b875 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:19.932406407 +0000 UTC m=+3.940271945 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret") pod "global-pull-secret-syncer-t5bf9" (UID: "dae75d3b-5868-4cd2-b7e4-45f6a145b875") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:18.943823 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:18.943773    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod939e5bee_11fa_4cc1_852b_49fba60700ee.slice/crio-1e7e74bc5c3ad314a30164849cc870d51267138a813174252b30315480651701 WatchSource:0}: Error finding container 1e7e74bc5c3ad314a30164849cc870d51267138a813174252b30315480651701: Status 404 returned error can't find the container with id 1e7e74bc5c3ad314a30164849cc870d51267138a813174252b30315480651701
Jul 28 22:04:18.973399 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:18.973369    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod769507e6_3089_4849_865d_283919d41c01.slice/crio-beba0a6069cd55939b4e8f4d77000d49229361cc85e31a2518edd22656f7b1f6 WatchSource:0}: Error finding container beba0a6069cd55939b4e8f4d77000d49229361cc85e31a2518edd22656f7b1f6: Status 404 returned error can't find the container with id beba0a6069cd55939b4e8f4d77000d49229361cc85e31a2518edd22656f7b1f6
Jul 28 22:04:18.973931 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:18.973902    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a85189c_c2ee_4a08_bb3e_7ad54914bc63.slice/crio-a735731a985162b636c19bea9ce4ffee2bdfc970cac02795f00b9e635e0e536c WatchSource:0}: Error finding container a735731a985162b636c19bea9ce4ffee2bdfc970cac02795f00b9e635e0e536c: Status 404 returned error can't find the container with id a735731a985162b636c19bea9ce4ffee2bdfc970cac02795f00b9e635e0e536c
Jul 28 22:04:18.975682 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:18.975213    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a8f707b_f715_437a_b41e_550b40dfe6c4.slice/crio-0df5a7bb28b9230322d0408dd3b5a8d1f3e8f540fa765671e594e1a94b6fa285 WatchSource:0}: Error finding container 0df5a7bb28b9230322d0408dd3b5a8d1f3e8f540fa765671e594e1a94b6fa285: Status 404 returned error can't find the container with id 0df5a7bb28b9230322d0408dd3b5a8d1f3e8f540fa765671e594e1a94b6fa285
Jul 28 22:04:18.976146 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:18.976062    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb0604f9_fa11_4595_9276_4430c567837b.slice/crio-adcebfd3240237a01e184573178b06e1ddd072b9171e8e1b1f6c95604f94495b WatchSource:0}: Error finding container adcebfd3240237a01e184573178b06e1ddd072b9171e8e1b1f6c95604f94495b: Status 404 returned error can't find the container with id adcebfd3240237a01e184573178b06e1ddd072b9171e8e1b1f6c95604f94495b
Jul 28 22:04:18.977410 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:18.977388    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod928ed8af_4599_402e_8dc6_5a3ca176d89f.slice/crio-dcae39a77c2117e005fd91c253afa8b2b54f255d75dc1ac334defd202e7839a1 WatchSource:0}: Error finding container dcae39a77c2117e005fd91c253afa8b2b54f255d75dc1ac334defd202e7839a1: Status 404 returned error can't find the container with id dcae39a77c2117e005fd91c253afa8b2b54f255d75dc1ac334defd202e7839a1
Jul 28 22:04:18.979602 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:18.978615    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24a96084_4f66_4d90_9de4_291369338782.slice/crio-cf7db2390ea89f573fa73606d1c931ece70a4cadb0c7ff4626c70a7dfaac9e11 WatchSource:0}: Error finding container cf7db2390ea89f573fa73606d1c931ece70a4cadb0c7ff4626c70a7dfaac9e11: Status 404 returned error can't find the container with id cf7db2390ea89f573fa73606d1c931ece70a4cadb0c7ff4626c70a7dfaac9e11
Jul 28 22:04:18.979602 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:18.979522    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4028222a_2ecd_4831_aded_68dde7adbdae.slice/crio-6a16c9c4a3453ac6dc90914bfb1e39ba5e0155c5420e6417c55dc7a24fd0cdad WatchSource:0}: Error finding container 6a16c9c4a3453ac6dc90914bfb1e39ba5e0155c5420e6417c55dc7a24fd0cdad: Status 404 returned error can't find the container with id 6a16c9c4a3453ac6dc90914bfb1e39ba5e0155c5420e6417c55dc7a24fd0cdad
Jul 28 22:04:18.982317 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:18.982289    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b7aaeb_cc08_452a_89b7_59cf107dce99.slice/crio-f5e7e4027f544c3bbf39e69b6b1339f862e91effe83cc4163f7b3257d414eefb WatchSource:0}: Error finding container f5e7e4027f544c3bbf39e69b6b1339f862e91effe83cc4163f7b3257d414eefb: Status 404 returned error can't find the container with id f5e7e4027f544c3bbf39e69b6b1339f862e91effe83cc4163f7b3257d414eefb
Jul 28 22:04:19.033257 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.033227    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk6vq\" (UniqueName: \"kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq\") pod \"network-check-target-btcz4\" (UID: \"b83ca49e-9d70-42ee-b256-2d4d995c598e\") " pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:19.033401 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.033286    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs\") pod \"network-metrics-daemon-5vh87\" (UID: \"33b4bdc7-694b-45e4-8c88-3cd9f134865d\") " pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:19.033401 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:19.033391    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:19.033517 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:19.033395    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:04:19.033517 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:19.033418    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:04:19.033517 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:19.033428    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kk6vq for pod openshift-network-diagnostics/network-check-target-btcz4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:19.033517 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:19.033449    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs podName:33b4bdc7-694b-45e4-8c88-3cd9f134865d nodeName:}" failed. No retries permitted until 2026-07-28 22:04:20.033433805 +0000 UTC m=+4.041299343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs") pod "network-metrics-daemon-5vh87" (UID: "33b4bdc7-694b-45e4-8c88-3cd9f134865d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:19.033517 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:19.033480    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq podName:b83ca49e-9d70-42ee-b256-2d4d995c598e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:20.033464701 +0000 UTC m=+4.041330258 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kk6vq" (UniqueName: "kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq") pod "network-check-target-btcz4" (UID: "b83ca49e-9d70-42ee-b256-2d4d995c598e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:19.362419 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.362329    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 21:59:18 +0000 UTC" deadline="2028-01-15 10:03:31.433284126 +0000 UTC"
Jul 28 22:04:19.362419 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.362379    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12851h59m12.070927504s"
Jul 28 22:04:19.435265 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.435231    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zvgg7" event={"ID":"63b7aaeb-cc08-452a-89b7-59cf107dce99","Type":"ContainerStarted","Data":"f5e7e4027f544c3bbf39e69b6b1339f862e91effe83cc4163f7b3257d414eefb"}
Jul 28 22:04:19.436626 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.436594    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2" event={"ID":"4028222a-2ecd-4831-aded-68dde7adbdae","Type":"ContainerStarted","Data":"6a16c9c4a3453ac6dc90914bfb1e39ba5e0155c5420e6417c55dc7a24fd0cdad"}
Jul 28 22:04:19.437882 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.437850    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v66pk" event={"ID":"24a96084-4f66-4d90-9de4-291369338782","Type":"ContainerStarted","Data":"cf7db2390ea89f573fa73606d1c931ece70a4cadb0c7ff4626c70a7dfaac9e11"}
Jul 28 22:04:19.439202 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.439177    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl" event={"ID":"7a8f707b-f715-437a-b41e-550b40dfe6c4","Type":"ContainerStarted","Data":"0df5a7bb28b9230322d0408dd3b5a8d1f3e8f540fa765671e594e1a94b6fa285"}
Jul 28 22:04:19.440488 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.440463    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjwtm" event={"ID":"2a85189c-c2ee-4a08-bb3e-7ad54914bc63","Type":"ContainerStarted","Data":"a735731a985162b636c19bea9ce4ffee2bdfc970cac02795f00b9e635e0e536c"}
Jul 28 22:04:19.441708 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.441686    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4hv59" event={"ID":"939e5bee-11fa-4cc1-852b-49fba60700ee","Type":"ContainerStarted","Data":"1e7e74bc5c3ad314a30164849cc870d51267138a813174252b30315480651701"}
Jul 28 22:04:19.443485 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.443458    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wqv74" event={"ID":"928ed8af-4599-402e-8dc6-5a3ca176d89f","Type":"ContainerStarted","Data":"dcae39a77c2117e005fd91c253afa8b2b54f255d75dc1ac334defd202e7839a1"}
Jul 28 22:04:19.444576 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.444551    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mgxvb" event={"ID":"db0604f9-fa11-4595-9276-4430c567837b","Type":"ContainerStarted","Data":"adcebfd3240237a01e184573178b06e1ddd072b9171e8e1b1f6c95604f94495b"}
Jul 28 22:04:19.445717 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.445698    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lkmdh" event={"ID":"769507e6-3089-4849-865d-283919d41c01","Type":"ContainerStarted","Data":"beba0a6069cd55939b4e8f4d77000d49229361cc85e31a2518edd22656f7b1f6"}
Jul 28 22:04:19.941614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:19.941392    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:19.941791 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:19.941758    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:19.941885 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:19.941825    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret podName:dae75d3b-5868-4cd2-b7e4-45f6a145b875 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:21.941807333 +0000 UTC m=+5.949672877 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret") pod "global-pull-secret-syncer-t5bf9" (UID: "dae75d3b-5868-4cd2-b7e4-45f6a145b875") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:20.042275 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:20.042242    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk6vq\" (UniqueName: \"kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq\") pod \"network-check-target-btcz4\" (UID: \"b83ca49e-9d70-42ee-b256-2d4d995c598e\") " pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:20.042403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:20.042316    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs\") pod \"network-metrics-daemon-5vh87\" (UID: \"33b4bdc7-694b-45e4-8c88-3cd9f134865d\") " pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:20.042470 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:20.042458    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:20.042524 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:20.042515    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs podName:33b4bdc7-694b-45e4-8c88-3cd9f134865d nodeName:}" failed. No retries permitted until 2026-07-28 22:04:22.042493865 +0000 UTC m=+6.050359405 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs") pod "network-metrics-daemon-5vh87" (UID: "33b4bdc7-694b-45e4-8c88-3cd9f134865d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:20.042616 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:20.042600    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:04:20.042683 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:20.042626    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:04:20.042683 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:20.042640    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kk6vq for pod openshift-network-diagnostics/network-check-target-btcz4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:20.042683 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:20.042678    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq podName:b83ca49e-9d70-42ee-b256-2d4d995c598e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:22.042664811 +0000 UTC m=+6.050530354 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kk6vq" (UniqueName: "kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq") pod "network-check-target-btcz4" (UID: "b83ca49e-9d70-42ee-b256-2d4d995c598e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:20.364484 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:20.364363    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 21:59:18 +0000 UTC" deadline="2028-02-26 05:18:20.093817768 +0000 UTC"
Jul 28 22:04:20.364484 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:20.364429    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13855h13m59.729392312s"
Jul 28 22:04:20.426340 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:20.425311    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:20.426340 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:20.425450    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:20.426340 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:20.425890    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:20.426340 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:20.425990    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:20.426340 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:20.426060    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:20.426340 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:20.426142    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:20.454379 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:20.454345    2583 generic.go:356] "Generic (PLEG): container finished" podID="919572f6e569012b430fa28bc75de12b" containerID="6f11ec0891537399e4002cea5787a54122c83ec6dbf31d05693978082460339d" exitCode=0
Jul 28 22:04:20.454554 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:20.454449    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal" event={"ID":"919572f6e569012b430fa28bc75de12b","Type":"ContainerDied","Data":"6f11ec0891537399e4002cea5787a54122c83ec6dbf31d05693978082460339d"}
Jul 28 22:04:20.468121 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:20.468091    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-9.ec2.internal" event={"ID":"41dbbb64873b28a94876546d4a4c4566","Type":"ContainerStarted","Data":"f3a8cbe110aa6db2fa17a4b6f9574e52c0fcbd2cd1391637cc3e3102e04e59d3"}
Jul 28 22:04:20.483859 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:20.483798    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-9.ec2.internal" podStartSLOduration=2.48377783 podStartE2EDuration="2.48377783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:20.483433515 +0000 UTC m=+4.491299100" watchObservedRunningTime="2026-07-28 22:04:20.48377783 +0000 UTC m=+4.491643393"
Jul 28 22:04:21.493204 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:21.493164    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal" event={"ID":"919572f6e569012b430fa28bc75de12b","Type":"ContainerStarted","Data":"cd509bb38af28c0754eb159a63840cdab6afbfd97d07672d3dc00b644d386edc"}
Jul 28 22:04:21.959652 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:21.959433    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:21.959823 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:21.959700    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:21.959823 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:21.959772    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret podName:dae75d3b-5868-4cd2-b7e4-45f6a145b875 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:25.959751371 +0000 UTC m=+9.967616911 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret") pod "global-pull-secret-syncer-t5bf9" (UID: "dae75d3b-5868-4cd2-b7e4-45f6a145b875") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:22.061175 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:22.060261    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs\") pod \"network-metrics-daemon-5vh87\" (UID: \"33b4bdc7-694b-45e4-8c88-3cd9f134865d\") " pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:22.061175 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:22.060355    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk6vq\" (UniqueName: \"kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq\") pod \"network-check-target-btcz4\" (UID: \"b83ca49e-9d70-42ee-b256-2d4d995c598e\") " pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:22.061175 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:22.060556    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:04:22.061175 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:22.060578    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:04:22.061175 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:22.060592    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kk6vq for pod openshift-network-diagnostics/network-check-target-btcz4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:22.061175 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:22.060654    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq podName:b83ca49e-9d70-42ee-b256-2d4d995c598e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:26.060634483 +0000 UTC m=+10.068500029 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kk6vq" (UniqueName: "kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq") pod "network-check-target-btcz4" (UID: "b83ca49e-9d70-42ee-b256-2d4d995c598e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:22.061175 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:22.061080    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:22.061175 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:22.061137    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs podName:33b4bdc7-694b-45e4-8c88-3cd9f134865d nodeName:}" failed. No retries permitted until 2026-07-28 22:04:26.061119633 +0000 UTC m=+10.068985179 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs") pod "network-metrics-daemon-5vh87" (UID: "33b4bdc7-694b-45e4-8c88-3cd9f134865d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:22.431611 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:22.430714    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:22.431611 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:22.430894    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:22.431611 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:22.431005    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:22.431611 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:22.431413    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:22.431611 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:22.431513    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:22.432071 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:22.432017    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:24.429582 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:24.429546    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:24.430081 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:24.429701    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:24.430158 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:24.430095    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:24.430380 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:24.430207    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:24.430380 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:24.430243    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:24.430380 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:24.430337    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:25.998468 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:25.998177    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:25.998468 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:25.998341    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:25.998468 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:25.998411    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret podName:dae75d3b-5868-4cd2-b7e4-45f6a145b875 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:33.998396643 +0000 UTC m=+18.006262182 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret") pod "global-pull-secret-syncer-t5bf9" (UID: "dae75d3b-5868-4cd2-b7e4-45f6a145b875") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:26.099595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:26.099561    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs\") pod \"network-metrics-daemon-5vh87\" (UID: \"33b4bdc7-694b-45e4-8c88-3cd9f134865d\") " pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:26.099765 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:26.099642    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk6vq\" (UniqueName: \"kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq\") pod \"network-check-target-btcz4\" (UID: \"b83ca49e-9d70-42ee-b256-2d4d995c598e\") " pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:26.099860 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:26.099767    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:26.099860 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:26.099778    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:04:26.099860 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:26.099817    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:04:26.099860 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:26.099831    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kk6vq for pod openshift-network-diagnostics/network-check-target-btcz4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:26.099860 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:26.099861    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs podName:33b4bdc7-694b-45e4-8c88-3cd9f134865d nodeName:}" failed. No retries permitted until 2026-07-28 22:04:34.099839826 +0000 UTC m=+18.107705379 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs") pod "network-metrics-daemon-5vh87" (UID: "33b4bdc7-694b-45e4-8c88-3cd9f134865d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:26.100096 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:26.099882    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq podName:b83ca49e-9d70-42ee-b256-2d4d995c598e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:34.099870845 +0000 UTC m=+18.107736389 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kk6vq" (UniqueName: "kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq") pod "network-check-target-btcz4" (UID: "b83ca49e-9d70-42ee-b256-2d4d995c598e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:26.427032 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:26.426921    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:26.427660 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:26.427237    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:26.427660 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:26.427395    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:26.427660 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:26.427490    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:26.427660 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:26.427263    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:26.427660 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:26.427621    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:28.429386 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:28.428905    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:28.429386 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:28.428942    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:28.429386 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:28.428911    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:28.429386 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:28.429038    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:28.429386 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:28.429084    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:28.429386 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:28.429145    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:30.425953 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:30.425866    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:30.425953 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:30.425898    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:30.426372 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:30.425865    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:30.426372 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:30.426004    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:30.426372 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:30.426134    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:30.426372 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:30.426234    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:32.425866 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:32.425820    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:32.425866 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:32.425841    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:32.426283 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:32.425906    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:32.426283 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:32.426016    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:32.426283 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:32.426153    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:32.426283 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:32.426267    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:34.061950 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:34.061912    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:34.062353 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:34.062029    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:34.062353 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:34.062083    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret podName:dae75d3b-5868-4cd2-b7e4-45f6a145b875 nodeName:}" failed. No retries permitted until 2026-07-28 22:04:50.062068462 +0000 UTC m=+34.069934001 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret") pod "global-pull-secret-syncer-t5bf9" (UID: "dae75d3b-5868-4cd2-b7e4-45f6a145b875") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:34.162917 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:34.162885    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk6vq\" (UniqueName: \"kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq\") pod \"network-check-target-btcz4\" (UID: \"b83ca49e-9d70-42ee-b256-2d4d995c598e\") " pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:34.163073 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:34.162938    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs\") pod \"network-metrics-daemon-5vh87\" (UID: \"33b4bdc7-694b-45e4-8c88-3cd9f134865d\") " pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:34.163073 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:34.163035    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:34.163073 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:34.163042    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:04:34.163073 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:34.163062    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:04:34.163073 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:34.163074    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kk6vq for pod openshift-network-diagnostics/network-check-target-btcz4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:34.163240 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:34.163100    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs podName:33b4bdc7-694b-45e4-8c88-3cd9f134865d nodeName:}" failed. No retries permitted until 2026-07-28 22:04:50.163080799 +0000 UTC m=+34.170946345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs") pod "network-metrics-daemon-5vh87" (UID: "33b4bdc7-694b-45e4-8c88-3cd9f134865d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:34.163240 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:34.163116    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq podName:b83ca49e-9d70-42ee-b256-2d4d995c598e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:50.16310846 +0000 UTC m=+34.170973999 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kk6vq" (UniqueName: "kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq") pod "network-check-target-btcz4" (UID: "b83ca49e-9d70-42ee-b256-2d4d995c598e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:34.425741 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:34.425657    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:34.425891 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:34.425657    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:34.425891 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:34.425806    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:34.425891 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:34.425824    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:34.426002 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:34.425938    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:34.426064 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:34.426037    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:36.426398 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:36.426367    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:36.426398 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:36.426409    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:36.426941 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:36.426479    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:36.426941 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:36.426507    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:36.426941 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:36.426606    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:36.426941 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:36.426675    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:38.426069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:38.426043    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:38.426425 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:38.426050    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:38.426425 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:38.426200    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:38.426646 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:38.426628    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:38.426749 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:38.426730    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:38.426851 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:38.426832    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:38.535227 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:38.535045    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lkmdh" event={"ID":"769507e6-3089-4849-865d-283919d41c01","Type":"ContainerStarted","Data":"ac2f5e7593a78c1f9f68e216a5bb7176c2a5e7277109d69543d90ff5ae005578"}
Jul 28 22:04:38.544356 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:38.544306    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zvgg7" event={"ID":"63b7aaeb-cc08-452a-89b7-59cf107dce99","Type":"ContainerStarted","Data":"ad885c6b5c4ee210e46a7da8c49346c6e85e044114fcf08957c22e8c2860cccb"}
Jul 28 22:04:38.548263 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:38.548210    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjwtm" event={"ID":"2a85189c-c2ee-4a08-bb3e-7ad54914bc63","Type":"ContainerStarted","Data":"172b40878fdde402e73262b64b4f9f2c82afcbd03e81cf89c5f04d616c7bba14"}
Jul 28 22:04:38.553248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:38.552604    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lkmdh" podStartSLOduration=6.297301785 podStartE2EDuration="21.552585752s" totalImagesPullingTime="15.255283967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:38.552036671 +0000 UTC m=+22.559902233" watchObservedRunningTime="2026-07-28 22:04:38.552585752 +0000 UTC m=+22.560451313"
Jul 28 22:04:38.553248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:38.552746    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-9.ec2.internal" podStartSLOduration=20.552739238 podStartE2EDuration="20.552739238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:21.508451053 +0000 UTC m=+5.516316617" watchObservedRunningTime="2026-07-28 22:04:38.552739238 +0000 UTC m=+22.560604798"
Jul 28 22:04:38.591255 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:38.591095    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zvgg7" podStartSLOduration=2.215325784 podStartE2EDuration="21.591075095s" totalImagesPullingTime="19.375749311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:38.59078005 +0000 UTC m=+22.598645609" watchObservedRunningTime="2026-07-28 22:04:38.591075095 +0000 UTC m=+22.598940655"
Jul 28 22:04:39.551740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.551573    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2" event={"ID":"4028222a-2ecd-4831-aded-68dde7adbdae","Type":"ContainerStarted","Data":"abdb69e15b5d8d88259f55807276498a951daa0723fba2d616458644c6bfc4a5"}
Jul 28 22:04:39.552811 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.552777    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v66pk" event={"ID":"24a96084-4f66-4d90-9de4-291369338782","Type":"ContainerStarted","Data":"57793c1c424dcf0b7fbd07577827a2a58ab5e631487ecd1362487b0dc97e857c"}
Jul 28 22:04:39.555410 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.555387    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl" event={"ID":"7a8f707b-f715-437a-b41e-550b40dfe6c4","Type":"ContainerStarted","Data":"227c68751f9d738c16c68e5cfd20702f8d88d2b064fa7f43f74d85e6a8a33ada"}
Jul 28 22:04:39.555500 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.555466    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl" event={"ID":"7a8f707b-f715-437a-b41e-550b40dfe6c4","Type":"ContainerStarted","Data":"2618a8d67cf66d7ebae21731af988e3a38e250f5c514f08372777a45ba3fb14e"}
Jul 28 22:04:39.555560 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.555521    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl" event={"ID":"7a8f707b-f715-437a-b41e-550b40dfe6c4","Type":"ContainerStarted","Data":"90e683ea4cd307927a0b3b8cd7be082a9075355926254fc24dc5563bc6a7edbe"}
Jul 28 22:04:39.555614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.555571    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl" event={"ID":"7a8f707b-f715-437a-b41e-550b40dfe6c4","Type":"ContainerStarted","Data":"1ee41144a2fb47ad4671f4bd1d07b879e20ef395bfc073985fa6a2d730a57c27"}
Jul 28 22:04:39.555670 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.555620    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl" event={"ID":"7a8f707b-f715-437a-b41e-550b40dfe6c4","Type":"ContainerStarted","Data":"00cf24e640ff5e0ff6e842164c3d55d0d871e6e9709df2924f9685aa171be407"}
Jul 28 22:04:39.555730 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.555668    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl" event={"ID":"7a8f707b-f715-437a-b41e-550b40dfe6c4","Type":"ContainerStarted","Data":"4e7e9a58b43e1b5bfe15dc467229c63dc7f5c448dd88738a44c37089bab098bd"}
Jul 28 22:04:39.556718 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.556693    2583 generic.go:356] "Generic (PLEG): container finished" podID="2a85189c-c2ee-4a08-bb3e-7ad54914bc63" containerID="172b40878fdde402e73262b64b4f9f2c82afcbd03e81cf89c5f04d616c7bba14" exitCode=0
Jul 28 22:04:39.556820 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.556775    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjwtm" event={"ID":"2a85189c-c2ee-4a08-bb3e-7ad54914bc63","Type":"ContainerDied","Data":"172b40878fdde402e73262b64b4f9f2c82afcbd03e81cf89c5f04d616c7bba14"}
Jul 28 22:04:39.558252 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.558169    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4hv59" event={"ID":"939e5bee-11fa-4cc1-852b-49fba60700ee","Type":"ContainerStarted","Data":"3f0149a8383dbb60040d3da4c960a904cf7eba00e67e9cf9e5ba0fc4c61c5034"}
Jul 28 22:04:39.559472 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.559447    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wqv74" event={"ID":"928ed8af-4599-402e-8dc6-5a3ca176d89f","Type":"ContainerStarted","Data":"21240f618999eb0b73972b9afe1810fe3aeade65f3dd7cad481313df078edf90"}
Jul 28 22:04:39.560779 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.560758    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mgxvb" event={"ID":"db0604f9-fa11-4595-9276-4430c567837b","Type":"ContainerStarted","Data":"a4d69edf03f66f1d131958ac305e566c649627a0dec704d0c40860fcd6bb5ae5"}
Jul 28 22:04:39.595240 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.595202    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mgxvb" podStartSLOduration=3.212260509 podStartE2EDuration="22.595190249s" totalImagesPullingTime="19.38292974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:39.594805629 +0000 UTC m=+23.602671178" watchObservedRunningTime="2026-07-28 22:04:39.595190249 +0000 UTC m=+23.603055806"
Jul 28 22:04:39.607238 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.607204    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4hv59" podStartSLOduration=3.217172323 podStartE2EDuration="22.607193496s" totalImagesPullingTime="19.390021173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:39.606950342 +0000 UTC m=+23.614815901" watchObservedRunningTime="2026-07-28 22:04:39.607193496 +0000 UTC m=+23.615059054"
Jul 28 22:04:39.634419 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.634388    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wqv74" podStartSLOduration=3.069564214 podStartE2EDuration="22.634376597s" totalImagesPullingTime="19.564812383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:39.634352116 +0000 UTC m=+23.642217675" watchObservedRunningTime="2026-07-28 22:04:39.634376597 +0000 UTC m=+23.642242155"
Jul 28 22:04:39.798108 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:39.798076    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 22:04:40.383811 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:40.383704    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T22:04:39.798100553Z","UUID":"f949fa63-4e3d-408d-9a4f-83eefd0b58a4","Handler":null,"Name":"","Endpoint":""}
Jul 28 22:04:40.385429 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:40.385408    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 22:04:40.385583 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:40.385446    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 22:04:40.426006 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:40.425982    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:40.426291 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:40.426089    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:40.426291 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:40.426098    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:40.426291 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:40.426203    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:40.426291 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:40.426258    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:40.426504 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:40.426331    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:40.564922 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:40.564890    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2" event={"ID":"4028222a-2ecd-4831-aded-68dde7adbdae","Type":"ContainerStarted","Data":"971eba8a0e7db709ea8d88e8cd8ea76e23d4c63a3e224970364d19269a4585d0"}
Jul 28 22:04:40.579773 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:40.579714    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-v66pk" podStartSLOduration=4.201804372 podStartE2EDuration="23.579696608s" totalImagesPullingTime="19.377892236s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:40.579469007 +0000 UTC m=+24.587334566" watchObservedRunningTime="2026-07-28 22:04:40.579696608 +0000 UTC m=+24.587562167"
Jul 28 22:04:41.572383 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:41.572282    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl" event={"ID":"7a8f707b-f715-437a-b41e-550b40dfe6c4","Type":"ContainerStarted","Data":"81dad9abdfda091a918f9cbcc357a918439ff1da76534ccec0769fe3edea9ce5"}
Jul 28 22:04:41.630863 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:41.630832    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4hv59"
Jul 28 22:04:41.631673 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:41.631652    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4hv59"
Jul 28 22:04:42.426228 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:42.425994    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:42.426407 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:42.426003    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:42.426407 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:42.426336    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:42.426407 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:42.426004    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:42.426558 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:42.426408    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:42.426558 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:42.426466    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:42.577604 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:42.577554    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2" event={"ID":"4028222a-2ecd-4831-aded-68dde7adbdae","Type":"ContainerStarted","Data":"1af9c78df22b8efbbc87dad61634192ca794e7cb4b3b8f1dbd6eb582d0be9eab"}
Jul 28 22:04:42.578072 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:42.577893    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4hv59"
Jul 28 22:04:42.578483 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:42.578466    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4hv59"
Jul 28 22:04:42.597365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:42.597326    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-khkk2" podStartSLOduration=3.144965453 podStartE2EDuration="25.59731131s" totalImagesPullingTime="22.452345857s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:42.597060839 +0000 UTC m=+26.604926397" watchObservedRunningTime="2026-07-28 22:04:42.59731131 +0000 UTC m=+26.605176867"
Jul 28 22:04:44.426192 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:44.426034    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:44.426761 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:44.426034    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:44.426761 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:44.426271    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:44.426761 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:44.426134    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:44.426761 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:44.426381    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:44.426761 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:44.426477    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:44.585362 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:44.585258    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl" event={"ID":"7a8f707b-f715-437a-b41e-550b40dfe6c4","Type":"ContainerStarted","Data":"e9c7cdb3441f4e6c122d4e776396e0612ae73713e4b0b14f56dda49f2dad8933"}
Jul 28 22:04:44.585585 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:44.585566    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:44.587161 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:44.587130    2583 generic.go:356] "Generic (PLEG): container finished" podID="2a85189c-c2ee-4a08-bb3e-7ad54914bc63" containerID="392f4a3fc1d67a6f0a5c00ec3f93f2b6f7877c32343ad0a8371d81eac6a49d6f" exitCode=0
Jul 28 22:04:44.587283 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:44.587198    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjwtm" event={"ID":"2a85189c-c2ee-4a08-bb3e-7ad54914bc63","Type":"ContainerDied","Data":"392f4a3fc1d67a6f0a5c00ec3f93f2b6f7877c32343ad0a8371d81eac6a49d6f"}
Jul 28 22:04:44.601755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:44.601734    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:44.613336 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:44.613296    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl" podStartSLOduration=8.219208227 podStartE2EDuration="27.613281629s" totalImagesPullingTime="19.394073402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:04:44.612783472 +0000 UTC m=+28.620649030" watchObservedRunningTime="2026-07-28 22:04:44.613281629 +0000 UTC m=+28.621147187"
Jul 28 22:04:45.589374 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:45.589340    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:45.589812 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:45.589458    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:45.604383 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:45.604358    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:04:46.323618 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:46.323529    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-btcz4"]
Jul 28 22:04:46.323775 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:46.323660    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:46.323868 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:46.323770    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:46.324287 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:46.324263    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t5bf9"]
Jul 28 22:04:46.324376 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:46.324352    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:46.324455 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:46.324436    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:46.326462 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:46.326432    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5vh87"]
Jul 28 22:04:46.326545 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:46.326536    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:46.326655 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:46.326636    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:46.592626 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:46.592544    2583 generic.go:356] "Generic (PLEG): container finished" podID="2a85189c-c2ee-4a08-bb3e-7ad54914bc63" containerID="c59a91c682a9c67e8affbf2a3ba433938107afa616dc8034ef7a1bbeffb83863" exitCode=0
Jul 28 22:04:46.593095 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:46.592635    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjwtm" event={"ID":"2a85189c-c2ee-4a08-bb3e-7ad54914bc63","Type":"ContainerDied","Data":"c59a91c682a9c67e8affbf2a3ba433938107afa616dc8034ef7a1bbeffb83863"}
Jul 28 22:04:47.425566 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:47.425541    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:47.425686 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:47.425641    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:48.425500 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:48.425257    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:48.425851 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:48.425311    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:48.425851 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:48.425532    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:48.425851 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:48.425590    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:48.599327 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:48.599296    2583 generic.go:356] "Generic (PLEG): container finished" podID="2a85189c-c2ee-4a08-bb3e-7ad54914bc63" containerID="15357a15485193234e64945fa2f7075e82f660098b01702332888b4a6989c0d4" exitCode=0
Jul 28 22:04:48.599505 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:48.599336    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjwtm" event={"ID":"2a85189c-c2ee-4a08-bb3e-7ad54914bc63","Type":"ContainerDied","Data":"15357a15485193234e64945fa2f7075e82f660098b01702332888b4a6989c0d4"}
Jul 28 22:04:49.425045 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:49.425016    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:49.425185 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:49.425115    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btcz4" podUID="b83ca49e-9d70-42ee-b256-2d4d995c598e"
Jul 28 22:04:50.082095 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:50.082066    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:50.082483 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:50.082241    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:50.082483 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:50.082319    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret podName:dae75d3b-5868-4cd2-b7e4-45f6a145b875 nodeName:}" failed. No retries permitted until 2026-07-28 22:05:22.082298568 +0000 UTC m=+66.090164127 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret") pod "global-pull-secret-syncer-t5bf9" (UID: "dae75d3b-5868-4cd2-b7e4-45f6a145b875") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:04:50.183341 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:50.183300    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:50.183501 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:50.183402    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs podName:33b4bdc7-694b-45e4-8c88-3cd9f134865d nodeName:}" failed. No retries permitted until 2026-07-28 22:05:22.183362741 +0000 UTC m=+66.191228282 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs") pod "network-metrics-daemon-5vh87" (UID: "33b4bdc7-694b-45e4-8c88-3cd9f134865d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:04:50.185757 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:50.183898    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs\") pod \"network-metrics-daemon-5vh87\" (UID: \"33b4bdc7-694b-45e4-8c88-3cd9f134865d\") " pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:50.185757 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:50.184001    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk6vq\" (UniqueName: \"kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq\") pod \"network-check-target-btcz4\" (UID: \"b83ca49e-9d70-42ee-b256-2d4d995c598e\") " pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:50.185757 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:50.184142    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:04:50.185757 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:50.184160    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:04:50.185757 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:50.184171    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kk6vq for pod openshift-network-diagnostics/network-check-target-btcz4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:50.185757 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:50.184222    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq podName:b83ca49e-9d70-42ee-b256-2d4d995c598e nodeName:}" failed. No retries permitted until 2026-07-28 22:05:22.184207692 +0000 UTC m=+66.192073249 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kk6vq" (UniqueName: "kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq") pod "network-check-target-btcz4" (UID: "b83ca49e-9d70-42ee-b256-2d4d995c598e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:04:50.426180 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:50.426110    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:50.426327 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:50.426250    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t5bf9" podUID="dae75d3b-5868-4cd2-b7e4-45f6a145b875"
Jul 28 22:04:50.426327 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:50.426282    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:50.426414 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:50.426340    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5vh87" podUID="33b4bdc7-694b-45e4-8c88-3cd9f134865d"
Jul 28 22:04:51.331252 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.331172    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-9.ec2.internal" event="NodeReady"
Jul 28 22:04:51.331855 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.331315    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 22:04:51.366726 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.366697    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-77hcw"]
Jul 28 22:04:51.371618 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.371590    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.372992 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.372949    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b9f75cc74-f72j8"]
Jul 28 22:04:51.376123 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.375912    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 28 22:04:51.376123 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.375946    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 22:04:51.376123 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.375915    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 28 22:04:51.376123 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.375913    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 22:04:51.376123 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.376020    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qjhd5\""
Jul 28 22:04:51.376765 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.376746    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"]
Jul 28 22:04:51.376977 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.376947    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.381008 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.380963    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"]
Jul 28 22:04:51.381116 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.381062    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:04:51.381731 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.381713    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 22:04:51.381883 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.381865    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 22:04:51.381980 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.381926    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 22:04:51.382445 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.382155    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2lgj6\""
Jul 28 22:04:51.383191 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.383169    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 28 22:04:51.383495 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.383472    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-55ztt\""
Jul 28 22:04:51.384624 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.384607    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 28 22:04:51.387712 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.385715    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-57xd4"]
Jul 28 22:04:51.387712 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.385898    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 28 22:04:51.387712 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.385976    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"
Jul 28 22:04:51.388958 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.388935    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 28 22:04:51.389268 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.389248    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-89kfz\""
Jul 28 22:04:51.389333 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.389314    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 28 22:04:51.389492 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.389474    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 22:04:51.389869 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.389516    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 28 22:04:51.390062 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.389252    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 28 22:04:51.390126 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.388945    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 22:04:51.391360 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.391331    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-th52b"]
Jul 28 22:04:51.391464 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.391446    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.393421 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.393399    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 22:04:51.393558 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.393543    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 22:04:51.396184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.396166    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 22:04:51.396460 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.396443    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 22:04:51.398041 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.398019    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-77hcw"]
Jul 28 22:04:51.398157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.398056    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-57xd4"]
Jul 28 22:04:51.398157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.398077    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"]
Jul 28 22:04:51.398157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.398096    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-4dj68"]
Jul 28 22:04:51.398328 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.398161    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-th52b"
Jul 28 22:04:51.398328 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.398162    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 22:04:51.403041 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.402932    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6597fb4cff-sdft5"]
Jul 28 22:04:51.403154 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.403139    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4dj68"
Jul 28 22:04:51.403320 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.403300    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 28 22:04:51.403498 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.403480    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-mzs9k\""
Jul 28 22:04:51.403594 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.403575    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-g22jr\""
Jul 28 22:04:51.404096 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.404079    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 28 22:04:51.405372 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.405354    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 22:04:51.406349 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.406319    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cnh56\""
Jul 28 22:04:51.406600 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.406341    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"]
Jul 28 22:04:51.406922 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.406405    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.407879 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.407859    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 22:04:51.409094 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.409076    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 22:04:51.409257 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.409238    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 22:04:51.409688 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.409115    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 28 22:04:51.409688 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.409245    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 28 22:04:51.409688 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.409502    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 28 22:04:51.409688 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.409639    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 22:04:51.409688 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.409677    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 28 22:04:51.410105 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.409756    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qwjr6\""
Jul 28 22:04:51.413045 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.412954    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"]
Jul 28 22:04:51.413612 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.413580    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"
Jul 28 22:04:51.415476 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.415457    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 28 22:04:51.416351 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.416332    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 28 22:04:51.416639 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.416332    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-lmmz6\""
Jul 28 22:04:51.416709 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.416666    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 28 22:04:51.416761 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.416705    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 28 22:04:51.417002 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.416970    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"]
Jul 28 22:04:51.417096 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.417046    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:04:51.420012 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.419995    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 28 22:04:51.420407 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.420389    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 28 22:04:51.423726 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.421997    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:04:51.423726 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.422534    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-cfr4m\""
Jul 28 22:04:51.425035 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.425016    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 22:04:51.425644 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.425616    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ch4rl\""
Jul 28 22:04:51.426012 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.425992    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 22:04:51.426405 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.426383    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 22:04:51.426505 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.426491    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"]
Jul 28 22:04:51.426625 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.426609    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cqbpk"]
Jul 28 22:04:51.429416 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.426842    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:04:51.431451 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.431435    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bnqzv\""
Jul 28 22:04:51.433413 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.433397    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hjds4"]
Jul 28 22:04:51.433592 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.433573    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:51.436268 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.436248    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8xbs4\""
Jul 28 22:04:51.436459 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.436443    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 22:04:51.436610 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.436593    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 22:04:51.436714 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.436695    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b9f75cc74-f72j8"]
Jul 28 22:04:51.436838 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.436811    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-th52b"]
Jul 28 22:04:51.436898 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.436843    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"]
Jul 28 22:04:51.436898 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.436862    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-4dj68"]
Jul 28 22:04:51.436898 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.436879    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"]
Jul 28 22:04:51.436898 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.436898    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hjds4"]
Jul 28 22:04:51.437084 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.437000    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hjds4"
Jul 28 22:04:51.437276 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.437262    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6597fb4cff-sdft5"]
Jul 28 22:04:51.437428 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.437412    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cqbpk"]
Jul 28 22:04:51.437496 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.437449    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"]
Jul 28 22:04:51.439118 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.439097    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 22:04:51.439199 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.439120    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 22:04:51.439326 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.439313    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 22:04:51.439391 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.439341    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zn58b\""
Jul 28 22:04:51.494215 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494189    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g22d\" (UniqueName: \"kubernetes.io/projected/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-kube-api-access-8g22d\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.494334 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494227    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzbfz\" (UniqueName: \"kubernetes.io/projected/3e9b3b23-0e89-4d3f-8f19-6830b0b68215-kube-api-access-pzbfz\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p\" (UID: \"3e9b3b23-0e89-4d3f-8f19-6830b0b68215\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"
Jul 28 22:04:51.494334 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494255    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e9b3b23-0e89-4d3f-8f19-6830b0b68215-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p\" (UID: \"3e9b3b23-0e89-4d3f-8f19-6830b0b68215\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"
Jul 28 22:04:51.494334 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494301    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1465e125-b955-4c39-bc95-67661bd1f3a3-tmp\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.494479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494337    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1465e125-b955-4c39-bc95-67661bd1f3a3-serving-cert\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.494479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494369    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/96283224-d894-4e35-868c-028c1f572acf-image-registry-private-configuration\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.494479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494400    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.494479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494417    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvw57\" (UniqueName: \"kubernetes.io/projected/1465e125-b955-4c39-bc95-67661bd1f3a3-kube-api-access-fvw57\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.494479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494455    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bz2q\" (UniqueName: \"kubernetes.io/projected/0fcee02e-7d41-4794-96bf-9d49280416c3-kube-api-access-9bz2q\") pod \"volume-data-source-validator-549c6d54cf-th52b\" (UID: \"0fcee02e-7d41-4794-96bf-9d49280416c3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-th52b"
Jul 28 22:04:51.494766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494492    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg9pg\" (UniqueName: \"kubernetes.io/projected/9d2c8128-4019-4802-8cb0-8307891fd7c5-kube-api-access-qg9pg\") pod \"service-ca-operator-5d6c75f84f-kb5jl\" (UID: \"9d2c8128-4019-4802-8cb0-8307891fd7c5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"
Jul 28 22:04:51.494766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494517    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/095ffcbc-5ca5-483e-b0d5-c919b4b9ab88-serving-cert\") pod \"console-operator-584dcb6b54-57xd4\" (UID: \"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88\") " pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.494766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494543    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-bound-sa-token\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.494766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494569    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/095ffcbc-5ca5-483e-b0d5-c919b4b9ab88-trusted-ca\") pod \"console-operator-584dcb6b54-57xd4\" (UID: \"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88\") " pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.494766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494616    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1465e125-b955-4c39-bc95-67661bd1f3a3-snapshots\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.494766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494642    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1465e125-b955-4c39-bc95-67661bd1f3a3-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.494766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494673    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/96283224-d894-4e35-868c-028c1f572acf-trusted-ca\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.494766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494697    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/095ffcbc-5ca5-483e-b0d5-c919b4b9ab88-config\") pod \"console-operator-584dcb6b54-57xd4\" (UID: \"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88\") " pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.494766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494722    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d2c8128-4019-4802-8cb0-8307891fd7c5-serving-cert\") pod \"service-ca-operator-5d6c75f84f-kb5jl\" (UID: \"9d2c8128-4019-4802-8cb0-8307891fd7c5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"
Jul 28 22:04:51.495124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494787    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.495124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494838    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h54wk\" (UniqueName: \"kubernetes.io/projected/145369c0-52c4-43e3-a1c3-154831f04b30-kube-api-access-h54wk\") pod \"network-check-source-7d88cb55d8-4dj68\" (UID: \"145369c0-52c4-43e3-a1c3-154831f04b30\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4dj68"
Jul 28 22:04:51.495124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494869    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-stats-auth\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.495124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494931    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4vwn\" (UniqueName: \"kubernetes.io/projected/7bc1592a-3b91-40a0-979c-2799511bcbda-kube-api-access-t4vwn\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:04:51.495124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494966    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/96283224-d894-4e35-868c-028c1f572acf-ca-trust-extracted\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.495124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.494984    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e9b3b23-0e89-4d3f-8f19-6830b0b68215-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p\" (UID: \"3e9b3b23-0e89-4d3f-8f19-6830b0b68215\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"
Jul 28 22:04:51.495124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.495009    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d2c8128-4019-4802-8cb0-8307891fd7c5-config\") pod \"service-ca-operator-5d6c75f84f-kb5jl\" (UID: \"9d2c8128-4019-4802-8cb0-8307891fd7c5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"
Jul 28 22:04:51.495124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.495030    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/96283224-d894-4e35-868c-028c1f572acf-installation-pull-secrets\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.495124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.495102    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.495431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.495135    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v69p\" (UniqueName: \"kubernetes.io/projected/095ffcbc-5ca5-483e-b0d5-c919b4b9ab88-kube-api-access-8v69p\") pod \"console-operator-584dcb6b54-57xd4\" (UID: \"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88\") " pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.495431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.495177    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-default-certificate\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.495431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.495200    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7bc1592a-3b91-40a0-979c-2799511bcbda-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:04:51.495431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.495215    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/96283224-d894-4e35-868c-028c1f572acf-registry-certificates\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.495431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.495237    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:04:51.495431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.495264    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1465e125-b955-4c39-bc95-67661bd1f3a3-service-ca-bundle\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.495431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.495289    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc8g7\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-kube-api-access-bc8g7\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.566345 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.566307    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:51.596541 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596468    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grx76\" (UniqueName: \"kubernetes.io/projected/ac77877d-96a7-41ed-b29d-6e3dccfdecf3-kube-api-access-grx76\") pod \"ingress-canary-hjds4\" (UID: \"ac77877d-96a7-41ed-b29d-6e3dccfdecf3\") " pod="openshift-ingress-canary/ingress-canary-hjds4"
Jul 28 22:04:51.596541 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596512    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg9pg\" (UniqueName: \"kubernetes.io/projected/9d2c8128-4019-4802-8cb0-8307891fd7c5-kube-api-access-qg9pg\") pod \"service-ca-operator-5d6c75f84f-kb5jl\" (UID: \"9d2c8128-4019-4802-8cb0-8307891fd7c5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"
Jul 28 22:04:51.596755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596539    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/095ffcbc-5ca5-483e-b0d5-c919b4b9ab88-serving-cert\") pod \"console-operator-584dcb6b54-57xd4\" (UID: \"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88\") " pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.596755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596564    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-bound-sa-token\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.596755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596584    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/095ffcbc-5ca5-483e-b0d5-c919b4b9ab88-trusted-ca\") pod \"console-operator-584dcb6b54-57xd4\" (UID: \"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88\") " pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.596755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596609    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f11e7df-616c-49a2-9071-0f32a5b25adf-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-sswtl\" (UID: \"4f11e7df-616c-49a2-9071-0f32a5b25adf\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:04:51.596755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596642    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1465e125-b955-4c39-bc95-67661bd1f3a3-snapshots\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.596755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596666    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1465e125-b955-4c39-bc95-67661bd1f3a3-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.596755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596687    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bn89p\" (UID: \"e31f7dc9-6732-44c5-a131-d9802a13613f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:04:51.596755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596716    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/96283224-d894-4e35-868c-028c1f572acf-trusted-ca\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.596755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/095ffcbc-5ca5-483e-b0d5-c919b4b9ab88-config\") pod \"console-operator-584dcb6b54-57xd4\" (UID: \"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88\") " pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.597208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596772    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d2c8128-4019-4802-8cb0-8307891fd7c5-serving-cert\") pod \"service-ca-operator-5d6c75f84f-kb5jl\" (UID: \"9d2c8128-4019-4802-8cb0-8307891fd7c5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"
Jul 28 22:04:51.597208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596841    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.597208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596880    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54wk\" (UniqueName: \"kubernetes.io/projected/145369c0-52c4-43e3-a1c3-154831f04b30-kube-api-access-h54wk\") pod \"network-check-source-7d88cb55d8-4dj68\" (UID: \"145369c0-52c4-43e3-a1c3-154831f04b30\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4dj68"
Jul 28 22:04:51.597208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596946    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sccb2\" (UniqueName: \"kubernetes.io/projected/8b67f675-10f2-4616-85c9-7feec90a410c-kube-api-access-sccb2\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:51.597208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596973    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-stats-auth\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.597208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.596997    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4vwn\" (UniqueName: \"kubernetes.io/projected/7bc1592a-3b91-40a0-979c-2799511bcbda-kube-api-access-t4vwn\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:04:51.597208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.597022    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/96283224-d894-4e35-868c-028c1f572acf-ca-trust-extracted\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.597208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.597040    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e9b3b23-0e89-4d3f-8f19-6830b0b68215-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p\" (UID: \"3e9b3b23-0e89-4d3f-8f19-6830b0b68215\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"
Jul 28 22:04:51.597208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.597071    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d2c8128-4019-4802-8cb0-8307891fd7c5-config\") pod \"service-ca-operator-5d6c75f84f-kb5jl\" (UID: \"9d2c8128-4019-4802-8cb0-8307891fd7c5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"
Jul 28 22:04:51.597208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.597091    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/96283224-d894-4e35-868c-028c1f572acf-installation-pull-secrets\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.597208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.597127    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sswtl\" (UID: \"4f11e7df-616c-49a2-9071-0f32a5b25adf\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:04:51.597208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.597156    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.597208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.597178    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v69p\" (UniqueName: \"kubernetes.io/projected/095ffcbc-5ca5-483e-b0d5-c919b4b9ab88-kube-api-access-8v69p\") pod \"console-operator-584dcb6b54-57xd4\" (UID: \"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88\") " pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.598104 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.597936    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-default-certificate\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.598227 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.598131    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7bc1592a-3b91-40a0-979c-2799511bcbda-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:04:51.599610 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.599495    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1465e125-b955-4c39-bc95-67661bd1f3a3-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.599851 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.599826    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/095ffcbc-5ca5-483e-b0d5-c919b4b9ab88-trusted-ca\") pod \"console-operator-584dcb6b54-57xd4\" (UID: \"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88\") " pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.600111 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.600088    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/095ffcbc-5ca5-483e-b0d5-c919b4b9ab88-config\") pod \"console-operator-584dcb6b54-57xd4\" (UID: \"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88\") " pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.600278 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.600260    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1465e125-b955-4c39-bc95-67661bd1f3a3-snapshots\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.600837 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.600814    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/96283224-d894-4e35-868c-028c1f572acf-registry-certificates\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.600942 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.600863    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac77877d-96a7-41ed-b29d-6e3dccfdecf3-cert\") pod \"ingress-canary-hjds4\" (UID: \"ac77877d-96a7-41ed-b29d-6e3dccfdecf3\") " pod="openshift-ingress-canary/ingress-canary-hjds4"
Jul 28 22:04:51.600942 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.600910    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:04:51.601044 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.600985    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1465e125-b955-4c39-bc95-67661bd1f3a3-service-ca-bundle\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.601044 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.601018    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 22:04:51.601149 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.601032    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc8g7\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-kube-api-access-bc8g7\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.601149 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.601146    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls podName:7bc1592a-3b91-40a0-979c-2799511bcbda nodeName:}" failed. No retries permitted until 2026-07-28 22:04:52.101125641 +0000 UTC m=+36.108991195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-97gkk" (UID: "7bc1592a-3b91-40a0-979c-2799511bcbda") : secret "cluster-monitoring-operator-tls" not found
Jul 28 22:04:51.601433 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.601407    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d2c8128-4019-4802-8cb0-8307891fd7c5-serving-cert\") pod \"service-ca-operator-5d6c75f84f-kb5jl\" (UID: \"9d2c8128-4019-4802-8cb0-8307891fd7c5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.601861    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.601941    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b67f675-10f2-4616-85c9-7feec90a410c-tmp-dir\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.602015    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c2vr\" (UniqueName: \"kubernetes.io/projected/e31f7dc9-6732-44c5-a131-d9802a13613f-kube-api-access-8c2vr\") pod \"cluster-samples-operator-6748489656-bn89p\" (UID: \"e31f7dc9-6732-44c5-a131-d9802a13613f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.602056    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8b67f675-10f2-4616-85c9-7feec90a410c-config-volume\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.602101    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g22d\" (UniqueName: \"kubernetes.io/projected/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-kube-api-access-8g22d\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.602133    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzbfz\" (UniqueName: \"kubernetes.io/projected/3e9b3b23-0e89-4d3f-8f19-6830b0b68215-kube-api-access-pzbfz\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p\" (UID: \"3e9b3b23-0e89-4d3f-8f19-6830b0b68215\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.602190    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e9b3b23-0e89-4d3f-8f19-6830b0b68215-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p\" (UID: \"3e9b3b23-0e89-4d3f-8f19-6830b0b68215\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.602193    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/96283224-d894-4e35-868c-028c1f572acf-ca-trust-extracted\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.602223    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1465e125-b955-4c39-bc95-67661bd1f3a3-tmp\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.602299    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.602349    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs podName:5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:52.102336926 +0000 UTC m=+36.110202462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs") pod "router-default-6597fb4cff-sdft5" (UID: "5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e") : secret "router-metrics-certs-default" not found
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.602771    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle podName:5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:52.10273961 +0000 UTC m=+36.110605165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle") pod "router-default-6597fb4cff-sdft5" (UID: "5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e") : configmap references non-existent config key: service-ca.crt
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.601862    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d2c8128-4019-4802-8cb0-8307891fd7c5-config\") pod \"service-ca-operator-5d6c75f84f-kb5jl\" (UID: \"9d2c8128-4019-4802-8cb0-8307891fd7c5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.602835    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1465e125-b955-4c39-bc95-67661bd1f3a3-serving-cert\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.604975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.603116    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/96283224-d894-4e35-868c-028c1f572acf-image-registry-private-configuration\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.605692 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.603371    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1465e125-b955-4c39-bc95-67661bd1f3a3-tmp\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.605692 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.604205    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/095ffcbc-5ca5-483e-b0d5-c919b4b9ab88-serving-cert\") pod \"console-operator-584dcb6b54-57xd4\" (UID: \"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88\") " pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.605692 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.604567    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7bc1592a-3b91-40a0-979c-2799511bcbda-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:04:51.605692 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.604625    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/96283224-d894-4e35-868c-028c1f572acf-trusted-ca\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.605692 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.604642    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/96283224-d894-4e35-868c-028c1f572acf-installation-pull-secrets\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.605692 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.605260    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.605692 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.605333    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvw57\" (UniqueName: \"kubernetes.io/projected/1465e125-b955-4c39-bc95-67661bd1f3a3-kube-api-access-fvw57\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.605692 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.605380    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bz2q\" (UniqueName: \"kubernetes.io/projected/0fcee02e-7d41-4794-96bf-9d49280416c3-kube-api-access-9bz2q\") pod \"volume-data-source-validator-549c6d54cf-th52b\" (UID: \"0fcee02e-7d41-4794-96bf-9d49280416c3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-th52b"
Jul 28 22:04:51.605692 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.605646    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:04:51.605692 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.605663    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b9f75cc74-f72j8: secret "image-registry-tls" not found
Jul 28 22:04:51.606145 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.605829    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls podName:96283224-d894-4e35-868c-028c1f572acf nodeName:}" failed. No retries permitted until 2026-07-28 22:04:52.105788222 +0000 UTC m=+36.113653760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls") pod "image-registry-7b9f75cc74-f72j8" (UID: "96283224-d894-4e35-868c-028c1f572acf") : secret "image-registry-tls" not found
Jul 28 22:04:51.607817 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.606222    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/96283224-d894-4e35-868c-028c1f572acf-registry-certificates\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.607817 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.606831    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e9b3b23-0e89-4d3f-8f19-6830b0b68215-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p\" (UID: \"3e9b3b23-0e89-4d3f-8f19-6830b0b68215\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"
Jul 28 22:04:51.607817 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.607372    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1465e125-b955-4c39-bc95-67661bd1f3a3-service-ca-bundle\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.610938 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.610561    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/96283224-d894-4e35-868c-028c1f572acf-image-registry-private-configuration\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.610938 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.610883    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e9b3b23-0e89-4d3f-8f19-6830b0b68215-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p\" (UID: \"3e9b3b23-0e89-4d3f-8f19-6830b0b68215\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"
Jul 28 22:04:51.611176 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.611088    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-stats-auth\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.611268 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.611250    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-default-certificate\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.612865 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.612823    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1465e125-b955-4c39-bc95-67661bd1f3a3-serving-cert\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.613290 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.613261    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg9pg\" (UniqueName: \"kubernetes.io/projected/9d2c8128-4019-4802-8cb0-8307891fd7c5-kube-api-access-qg9pg\") pod \"service-ca-operator-5d6c75f84f-kb5jl\" (UID: \"9d2c8128-4019-4802-8cb0-8307891fd7c5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"
Jul 28 22:04:51.616133 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.615181    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvw57\" (UniqueName: \"kubernetes.io/projected/1465e125-b955-4c39-bc95-67661bd1f3a3-kube-api-access-fvw57\") pod \"insights-operator-5885fdb759-77hcw\" (UID: \"1465e125-b955-4c39-bc95-67661bd1f3a3\") " pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.616133 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.615240    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc8g7\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-kube-api-access-bc8g7\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.616133 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.615276    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4vwn\" (UniqueName: \"kubernetes.io/projected/7bc1592a-3b91-40a0-979c-2799511bcbda-kube-api-access-t4vwn\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:04:51.616133 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.615282    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h54wk\" (UniqueName: \"kubernetes.io/projected/145369c0-52c4-43e3-a1c3-154831f04b30-kube-api-access-h54wk\") pod \"network-check-source-7d88cb55d8-4dj68\" (UID: \"145369c0-52c4-43e3-a1c3-154831f04b30\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4dj68"
Jul 28 22:04:51.616133 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.615699    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bz2q\" (UniqueName: \"kubernetes.io/projected/0fcee02e-7d41-4794-96bf-9d49280416c3-kube-api-access-9bz2q\") pod \"volume-data-source-validator-549c6d54cf-th52b\" (UID: \"0fcee02e-7d41-4794-96bf-9d49280416c3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-th52b"
Jul 28 22:04:51.616133 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.615704    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v69p\" (UniqueName: \"kubernetes.io/projected/095ffcbc-5ca5-483e-b0d5-c919b4b9ab88-kube-api-access-8v69p\") pod \"console-operator-584dcb6b54-57xd4\" (UID: \"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88\") " pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.616133 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.616097    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-bound-sa-token\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:51.616518 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.616177    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g22d\" (UniqueName: \"kubernetes.io/projected/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-kube-api-access-8g22d\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:51.616518 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.616404    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzbfz\" (UniqueName: \"kubernetes.io/projected/3e9b3b23-0e89-4d3f-8f19-6830b0b68215-kube-api-access-pzbfz\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p\" (UID: \"3e9b3b23-0e89-4d3f-8f19-6830b0b68215\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"
Jul 28 22:04:51.670162 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.670115    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:51.688892 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.688868    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-77hcw"
Jul 28 22:04:51.706067 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.706026    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sccb2\" (UniqueName: \"kubernetes.io/projected/8b67f675-10f2-4616-85c9-7feec90a410c-kube-api-access-sccb2\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:51.706174 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.706082    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sswtl\" (UID: \"4f11e7df-616c-49a2-9071-0f32a5b25adf\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:04:51.706174 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.706128    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac77877d-96a7-41ed-b29d-6e3dccfdecf3-cert\") pod \"ingress-canary-hjds4\" (UID: \"ac77877d-96a7-41ed-b29d-6e3dccfdecf3\") " pod="openshift-ingress-canary/ingress-canary-hjds4"
Jul 28 22:04:51.706174 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.706160    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:51.706174 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.706175    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b67f675-10f2-4616-85c9-7feec90a410c-tmp-dir\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:51.706371 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.706199    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c2vr\" (UniqueName: \"kubernetes.io/projected/e31f7dc9-6732-44c5-a131-d9802a13613f-kube-api-access-8c2vr\") pod \"cluster-samples-operator-6748489656-bn89p\" (UID: \"e31f7dc9-6732-44c5-a131-d9802a13613f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:04:51.706371 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.706239    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8b67f675-10f2-4616-85c9-7feec90a410c-config-volume\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:51.706371 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.706271    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 22:04:51.706371 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.706302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grx76\" (UniqueName: \"kubernetes.io/projected/ac77877d-96a7-41ed-b29d-6e3dccfdecf3-kube-api-access-grx76\") pod \"ingress-canary-hjds4\" (UID: \"ac77877d-96a7-41ed-b29d-6e3dccfdecf3\") " pod="openshift-ingress-canary/ingress-canary-hjds4"
Jul 28 22:04:51.706371 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.706342    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert podName:4f11e7df-616c-49a2-9071-0f32a5b25adf nodeName:}" failed. No retries permitted until 2026-07-28 22:04:52.206319301 +0000 UTC m=+36.214184852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sswtl" (UID: "4f11e7df-616c-49a2-9071-0f32a5b25adf") : secret "networking-console-plugin-cert" not found
Jul 28 22:04:51.706615 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.706400    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f11e7df-616c-49a2-9071-0f32a5b25adf-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-sswtl\" (UID: \"4f11e7df-616c-49a2-9071-0f32a5b25adf\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:04:51.706615 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.706448    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bn89p\" (UID: \"e31f7dc9-6732-44c5-a131-d9802a13613f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:04:51.706615 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.706491    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b67f675-10f2-4616-85c9-7feec90a410c-tmp-dir\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:51.706615 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.706544    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 22:04:51.706615 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.706279    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:04:51.706615 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.706586    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls podName:e31f7dc9-6732-44c5-a131-d9802a13613f nodeName:}" failed. No retries permitted until 2026-07-28 22:04:52.206572585 +0000 UTC m=+36.214438123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls") pod "cluster-samples-operator-6748489656-bn89p" (UID: "e31f7dc9-6732-44c5-a131-d9802a13613f") : secret "samples-operator-tls" not found
Jul 28 22:04:51.706615 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:51.706617    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls podName:8b67f675-10f2-4616-85c9-7feec90a410c nodeName:}" failed. No retries permitted until 2026-07-28 22:04:52.206600502 +0000 UTC m=+36.214466045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls") pod "dns-default-cqbpk" (UID: "8b67f675-10f2-4616-85c9-7feec90a410c") : secret "dns-default-metrics-tls" not found
Jul 28 22:04:51.707148 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.707108    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8b67f675-10f2-4616-85c9-7feec90a410c-config-volume\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:51.707250 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.707147    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f11e7df-616c-49a2-9071-0f32a5b25adf-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-sswtl\" (UID: \"4f11e7df-616c-49a2-9071-0f32a5b25adf\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:04:51.708981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.708956    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac77877d-96a7-41ed-b29d-6e3dccfdecf3-cert\") pod \"ingress-canary-hjds4\" (UID: \"ac77877d-96a7-41ed-b29d-6e3dccfdecf3\") " pod="openshift-ingress-canary/ingress-canary-hjds4"
Jul 28 22:04:51.714679 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.714656    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sccb2\" (UniqueName: \"kubernetes.io/projected/8b67f675-10f2-4616-85c9-7feec90a410c-kube-api-access-sccb2\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:51.716055 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.716034    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"
Jul 28 22:04:51.716962 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.716940    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grx76\" (UniqueName: \"kubernetes.io/projected/ac77877d-96a7-41ed-b29d-6e3dccfdecf3-kube-api-access-grx76\") pod \"ingress-canary-hjds4\" (UID: \"ac77877d-96a7-41ed-b29d-6e3dccfdecf3\") " pod="openshift-ingress-canary/ingress-canary-hjds4"
Jul 28 22:04:51.717037 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.716992    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c2vr\" (UniqueName: \"kubernetes.io/projected/e31f7dc9-6732-44c5-a131-d9802a13613f-kube-api-access-8c2vr\") pod \"cluster-samples-operator-6748489656-bn89p\" (UID: \"e31f7dc9-6732-44c5-a131-d9802a13613f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:04:51.731448 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.731418    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:04:51.737268 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.737254    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-th52b"
Jul 28 22:04:51.741961 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.741946    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4dj68"
Jul 28 22:04:51.755866 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.755849    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"
Jul 28 22:04:51.788241 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:51.788219    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hjds4"
Jul 28 22:04:52.110652 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.110624    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:52.110854 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.110668    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:04:52.110854 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.110841    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle podName:5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:53.110818063 +0000 UTC m=+37.118683611 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle") pod "router-default-6597fb4cff-sdft5" (UID: "5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e") : configmap references non-existent config key: service-ca.crt
Jul 28 22:04:52.110987 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.110881    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 22:04:52.110987 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.110892    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:52.110987 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.110938    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls podName:7bc1592a-3b91-40a0-979c-2799511bcbda nodeName:}" failed. No retries permitted until 2026-07-28 22:04:53.110925798 +0000 UTC m=+37.118791348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-97gkk" (UID: "7bc1592a-3b91-40a0-979c-2799511bcbda") : secret "cluster-monitoring-operator-tls" not found
Jul 28 22:04:52.111148 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.111002    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:04:52.111148 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.111004    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:52.111148 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.111014    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b9f75cc74-f72j8: secret "image-registry-tls" not found
Jul 28 22:04:52.111148 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.111053    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls podName:96283224-d894-4e35-868c-028c1f572acf nodeName:}" failed. No retries permitted until 2026-07-28 22:04:53.111038439 +0000 UTC m=+37.118903977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls") pod "image-registry-7b9f75cc74-f72j8" (UID: "96283224-d894-4e35-868c-028c1f572acf") : secret "image-registry-tls" not found
Jul 28 22:04:52.111148 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.111105    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 22:04:52.111148 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.111147    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs podName:5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:53.11113461 +0000 UTC m=+37.119000157 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs") pod "router-default-6597fb4cff-sdft5" (UID: "5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e") : secret "router-metrics-certs-default" not found
Jul 28 22:04:52.212400 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.212368    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bn89p\" (UID: \"e31f7dc9-6732-44c5-a131-d9802a13613f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:04:52.212580 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.212470    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sswtl\" (UID: \"4f11e7df-616c-49a2-9071-0f32a5b25adf\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:04:52.212580 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.212494    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 22:04:52.212580 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.212531    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:52.212580 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.212566    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls podName:e31f7dc9-6732-44c5-a131-d9802a13613f nodeName:}" failed. No retries permitted until 2026-07-28 22:04:53.212545272 +0000 UTC m=+37.220410811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls") pod "cluster-samples-operator-6748489656-bn89p" (UID: "e31f7dc9-6732-44c5-a131-d9802a13613f") : secret "samples-operator-tls" not found
Jul 28 22:04:52.212781 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.212609    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 22:04:52.212781 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.212625    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:04:52.212781 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.212670    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls podName:8b67f675-10f2-4616-85c9-7feec90a410c nodeName:}" failed. No retries permitted until 2026-07-28 22:04:53.212656154 +0000 UTC m=+37.220521706 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls") pod "dns-default-cqbpk" (UID: "8b67f675-10f2-4616-85c9-7feec90a410c") : secret "dns-default-metrics-tls" not found
Jul 28 22:04:52.212781 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:52.212765    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert podName:4f11e7df-616c-49a2-9071-0f32a5b25adf nodeName:}" failed. No retries permitted until 2026-07-28 22:04:53.212749805 +0000 UTC m=+37.220615348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sswtl" (UID: "4f11e7df-616c-49a2-9071-0f32a5b25adf") : secret "networking-console-plugin-cert" not found
Jul 28 22:04:52.425699 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.425614    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:04:52.425699 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.425660    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:04:52.429066 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.429044    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 22:04:52.429411 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.429394    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 22:04:52.429491 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.429452    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sm6xq\""
Jul 28 22:04:52.793159 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.792482    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl"]
Jul 28 22:04:52.797487 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.797410    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:52.799557 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:52.799373    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2c8128_4019_4802_8cb0_8307891fd7c5.slice/crio-85bab42998facf848f6295522e142a0268ce6bffa76dd79c4e0b26ec93f566cf WatchSource:0}: Error finding container 85bab42998facf848f6295522e142a0268ce6bffa76dd79c4e0b26ec93f566cf: Status 404 returned error can't find the container with id 85bab42998facf848f6295522e142a0268ce6bffa76dd79c4e0b26ec93f566cf
Jul 28 22:04:52.819483 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.819453    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-4dj68"]
Jul 28 22:04:52.821679 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.821170    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-77hcw"]
Jul 28 22:04:52.825242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.822440    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hjds4"]
Jul 28 22:04:52.825242 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:52.824578    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1465e125_b955_4c39_bc95_67661bd1f3a3.slice/crio-5ac4dd2af7fca9bfa3c6ea995b83ea4278d6425732f93da3df5579f061dfaacc WatchSource:0}: Error finding container 5ac4dd2af7fca9bfa3c6ea995b83ea4278d6425732f93da3df5579f061dfaacc: Status 404 returned error can't find the container with id 5ac4dd2af7fca9bfa3c6ea995b83ea4278d6425732f93da3df5579f061dfaacc
Jul 28 22:04:52.825242 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:52.825140    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod145369c0_52c4_43e3_a1c3_154831f04b30.slice/crio-5ac67d3e15d4c85b1e98455b1de5e40607d05c9d4fe8f4ba341139cedc675b57 WatchSource:0}: Error finding container 5ac67d3e15d4c85b1e98455b1de5e40607d05c9d4fe8f4ba341139cedc675b57: Status 404 returned error can't find the container with id 5ac67d3e15d4c85b1e98455b1de5e40607d05c9d4fe8f4ba341139cedc675b57
Jul 28 22:04:52.825938 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.825909    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-57xd4"]
Jul 28 22:04:52.827170 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.827083    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p"]
Jul 28 22:04:52.838665 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.838641    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-th52b"]
Jul 28 22:04:52.842987 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:04:52.842955    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fcee02e_7d41_4794_96bf_9d49280416c3.slice/crio-b28533083c80eee5deaa3e39dd7400ac9a7dd6fbafe1f3788402ecd61cc93af9 WatchSource:0}: Error finding container b28533083c80eee5deaa3e39dd7400ac9a7dd6fbafe1f3788402ecd61cc93af9: Status 404 returned error can't find the container with id b28533083c80eee5deaa3e39dd7400ac9a7dd6fbafe1f3788402ecd61cc93af9
Jul 28 22:04:52.879749 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.879708    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:52.976013 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:52.975946    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:53.122135 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.122102    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:53.122276 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.122171    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:53.122276 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.122198    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:04:53.122276 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.122252    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 22:04:53.122276 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.122273    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:53.122419 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.122312    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs podName:5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:55.122297474 +0000 UTC m=+39.130163009 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs") pod "router-default-6597fb4cff-sdft5" (UID: "5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e") : secret "router-metrics-certs-default" not found
Jul 28 22:04:53.122419 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.122342    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle podName:5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:55.1223241 +0000 UTC m=+39.130189663 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle") pod "router-default-6597fb4cff-sdft5" (UID: "5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e") : configmap references non-existent config key: service-ca.crt
Jul 28 22:04:53.122419 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.122368    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:04:53.122419 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.122380    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b9f75cc74-f72j8: secret "image-registry-tls" not found
Jul 28 22:04:53.122419 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.122388    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 22:04:53.122585 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.122423    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls podName:96283224-d894-4e35-868c-028c1f572acf nodeName:}" failed. No retries permitted until 2026-07-28 22:04:55.122412739 +0000 UTC m=+39.130278277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls") pod "image-registry-7b9f75cc74-f72j8" (UID: "96283224-d894-4e35-868c-028c1f572acf") : secret "image-registry-tls" not found
Jul 28 22:04:53.122585 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.122455    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls podName:7bc1592a-3b91-40a0-979c-2799511bcbda nodeName:}" failed. No retries permitted until 2026-07-28 22:04:55.122437022 +0000 UTC m=+39.130302560 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-97gkk" (UID: "7bc1592a-3b91-40a0-979c-2799511bcbda") : secret "cluster-monitoring-operator-tls" not found
Jul 28 22:04:53.223191 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.223163    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sswtl\" (UID: \"4f11e7df-616c-49a2-9071-0f32a5b25adf\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:04:53.223341 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.223222    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:53.223341 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.223279    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bn89p\" (UID: \"e31f7dc9-6732-44c5-a131-d9802a13613f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:04:53.223341 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.223309    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 22:04:53.223430 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.223362    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert podName:4f11e7df-616c-49a2-9071-0f32a5b25adf nodeName:}" failed. No retries permitted until 2026-07-28 22:04:55.223349456 +0000 UTC m=+39.231214991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sswtl" (UID: "4f11e7df-616c-49a2-9071-0f32a5b25adf") : secret "networking-console-plugin-cert" not found
Jul 28 22:04:53.223430 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.223365    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 22:04:53.223430 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.223406    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls podName:e31f7dc9-6732-44c5-a131-d9802a13613f nodeName:}" failed. No retries permitted until 2026-07-28 22:04:55.223395854 +0000 UTC m=+39.231261390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls") pod "cluster-samples-operator-6748489656-bn89p" (UID: "e31f7dc9-6732-44c5-a131-d9802a13613f") : secret "samples-operator-tls" not found
Jul 28 22:04:53.223430 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.223365    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:04:53.223430 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:53.223432    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls podName:8b67f675-10f2-4616-85c9-7feec90a410c nodeName:}" failed. No retries permitted until 2026-07-28 22:04:55.223426434 +0000 UTC m=+39.231291970 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls") pod "dns-default-cqbpk" (UID: "8b67f675-10f2-4616-85c9-7feec90a410c") : secret "dns-default-metrics-tls" not found
Jul 28 22:04:53.619225 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.619131    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hjds4" event={"ID":"ac77877d-96a7-41ed-b29d-6e3dccfdecf3","Type":"ContainerStarted","Data":"11d7a245bc5dbc11211862e618033af956187f17814f7f0a960f9d21ab41d124"}
Jul 28 22:04:53.621520 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.621442    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl" event={"ID":"9d2c8128-4019-4802-8cb0-8307891fd7c5","Type":"ContainerStarted","Data":"85bab42998facf848f6295522e142a0268ce6bffa76dd79c4e0b26ec93f566cf"}
Jul 28 22:04:53.624216 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.624190    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4" event={"ID":"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88","Type":"ContainerStarted","Data":"6b7718c41e9ede898aead31accaf426bd10d501eb059c25bacb092b5badc8461"}
Jul 28 22:04:53.627070 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.626449    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4dj68" event={"ID":"145369c0-52c4-43e3-a1c3-154831f04b30","Type":"ContainerStarted","Data":"5ac67d3e15d4c85b1e98455b1de5e40607d05c9d4fe8f4ba341139cedc675b57"}
Jul 28 22:04:53.628447 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.628391    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-th52b" event={"ID":"0fcee02e-7d41-4794-96bf-9d49280416c3","Type":"ContainerStarted","Data":"b28533083c80eee5deaa3e39dd7400ac9a7dd6fbafe1f3788402ecd61cc93af9"}
Jul 28 22:04:53.629944 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.629878    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p" event={"ID":"3e9b3b23-0e89-4d3f-8f19-6830b0b68215","Type":"ContainerStarted","Data":"3883f5d71fb81e108117e86b822217bc5382cb0a931ab12e8299dafa1b1b4e7b"}
Jul 28 22:04:53.637570 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.636600    2583 generic.go:356] "Generic (PLEG): container finished" podID="2a85189c-c2ee-4a08-bb3e-7ad54914bc63" containerID="e0bd0c4336ca0935b445fc7601283f2d5c2d7aebf03bc8949af823e00e599067" exitCode=0
Jul 28 22:04:53.637570 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.636670    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjwtm" event={"ID":"2a85189c-c2ee-4a08-bb3e-7ad54914bc63","Type":"ContainerDied","Data":"e0bd0c4336ca0935b445fc7601283f2d5c2d7aebf03bc8949af823e00e599067"}
Jul 28 22:04:53.640209 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.640077    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-77hcw" event={"ID":"1465e125-b955-4c39-bc95-67661bd1f3a3","Type":"ContainerStarted","Data":"5ac4dd2af7fca9bfa3c6ea995b83ea4278d6425732f93da3df5579f061dfaacc"}
Jul 28 22:04:53.880535 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.880421    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:53.978562 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:53.978528    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:54.655019 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:54.654076    2583 generic.go:356] "Generic (PLEG): container finished" podID="2a85189c-c2ee-4a08-bb3e-7ad54914bc63" containerID="943095c3ed778bc34327837030c5003bdda10f21f28496755a2c4f6d7ab340b4" exitCode=0
Jul 28 22:04:54.655019 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:54.654142    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjwtm" event={"ID":"2a85189c-c2ee-4a08-bb3e-7ad54914bc63","Type":"ContainerDied","Data":"943095c3ed778bc34327837030c5003bdda10f21f28496755a2c4f6d7ab340b4"}
Jul 28 22:04:55.142132 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:55.142097    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:55.142304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:55.142156    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:04:55.142304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:55.142220    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:55.142442 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:55.142307    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:55.142503 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.142445    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 22:04:55.142555 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.142505    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs podName:5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:59.142486359 +0000 UTC m=+43.150351899 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs") pod "router-default-6597fb4cff-sdft5" (UID: "5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e") : secret "router-metrics-certs-default" not found
Jul 28 22:04:55.143186 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.142983    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle podName:5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e nodeName:}" failed. No retries permitted until 2026-07-28 22:04:59.142965476 +0000 UTC m=+43.150831018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle") pod "router-default-6597fb4cff-sdft5" (UID: "5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e") : configmap references non-existent config key: service-ca.crt
Jul 28 22:04:55.143186 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.142984    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:04:55.143186 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.143009    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b9f75cc74-f72j8: secret "image-registry-tls" not found
Jul 28 22:04:55.143186 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.143035    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls podName:96283224-d894-4e35-868c-028c1f572acf nodeName:}" failed. No retries permitted until 2026-07-28 22:04:59.143025361 +0000 UTC m=+43.150890898 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls") pod "image-registry-7b9f75cc74-f72j8" (UID: "96283224-d894-4e35-868c-028c1f572acf") : secret "image-registry-tls" not found
Jul 28 22:04:55.143186 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.143057    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 22:04:55.143186 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.143123    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls podName:7bc1592a-3b91-40a0-979c-2799511bcbda nodeName:}" failed. No retries permitted until 2026-07-28 22:04:59.143106707 +0000 UTC m=+43.150972246 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-97gkk" (UID: "7bc1592a-3b91-40a0-979c-2799511bcbda") : secret "cluster-monitoring-operator-tls" not found
Jul 28 22:04:55.243267 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:55.243219    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bn89p\" (UID: \"e31f7dc9-6732-44c5-a131-d9802a13613f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:04:55.243429 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:55.243327    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sswtl\" (UID: \"4f11e7df-616c-49a2-9071-0f32a5b25adf\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:04:55.243429 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:55.243395    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:55.243551 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.243492    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 22:04:55.243601 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.243551    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:04:55.243601 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.243569    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls podName:e31f7dc9-6732-44c5-a131-d9802a13613f nodeName:}" failed. No retries permitted until 2026-07-28 22:04:59.243548429 +0000 UTC m=+43.251413981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls") pod "cluster-samples-operator-6748489656-bn89p" (UID: "e31f7dc9-6732-44c5-a131-d9802a13613f") : secret "samples-operator-tls" not found
Jul 28 22:04:55.243601 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.243600    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls podName:8b67f675-10f2-4616-85c9-7feec90a410c nodeName:}" failed. No retries permitted until 2026-07-28 22:04:59.243585029 +0000 UTC m=+43.251450569 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls") pod "dns-default-cqbpk" (UID: "8b67f675-10f2-4616-85c9-7feec90a410c") : secret "dns-default-metrics-tls" not found
Jul 28 22:04:55.243756 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.243650    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 22:04:55.243756 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:55.243673    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert podName:4f11e7df-616c-49a2-9071-0f32a5b25adf nodeName:}" failed. No retries permitted until 2026-07-28 22:04:59.243666932 +0000 UTC m=+43.251532468 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sswtl" (UID: "4f11e7df-616c-49a2-9071-0f32a5b25adf") : secret "networking-console-plugin-cert" not found
Jul 28 22:04:56.736023 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:56.735990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:57.019730 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:57.019609    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:57.120441 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:57.120405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:58.407570 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:58.407529    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:58.507308 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:58.507281    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:58.666603 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:58.666517    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:58.734550 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:58.734516    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:59.184071 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:59.184033    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:59.184251 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:59.184121    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:04:59.184251 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:59.184159    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:04:59.184251 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.184191    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 22:04:59.184251 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:59.184244    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:04:59.184419 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.184272    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle podName:5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e nodeName:}" failed. No retries permitted until 2026-07-28 22:05:07.184254995 +0000 UTC m=+51.192120535 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle") pod "router-default-6597fb4cff-sdft5" (UID: "5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e") : configmap references non-existent config key: service-ca.crt
Jul 28 22:04:59.184419 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.184302    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs podName:5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e nodeName:}" failed. No retries permitted until 2026-07-28 22:05:07.18428484 +0000 UTC m=+51.192150380 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs") pod "router-default-6597fb4cff-sdft5" (UID: "5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e") : secret "router-metrics-certs-default" not found
Jul 28 22:04:59.184419 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.184328    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 22:04:59.184419 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.184380    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:04:59.184419 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.184398    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b9f75cc74-f72j8: secret "image-registry-tls" not found
Jul 28 22:04:59.184419 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.184402    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls podName:7bc1592a-3b91-40a0-979c-2799511bcbda nodeName:}" failed. No retries permitted until 2026-07-28 22:05:07.184387104 +0000 UTC m=+51.192252655 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-97gkk" (UID: "7bc1592a-3b91-40a0-979c-2799511bcbda") : secret "cluster-monitoring-operator-tls" not found
Jul 28 22:04:59.184623 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.184454    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls podName:96283224-d894-4e35-868c-028c1f572acf nodeName:}" failed. No retries permitted until 2026-07-28 22:05:07.184438911 +0000 UTC m=+51.192304461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls") pod "image-registry-7b9f75cc74-f72j8" (UID: "96283224-d894-4e35-868c-028c1f572acf") : secret "image-registry-tls" not found
Jul 28 22:04:59.285035 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:59.284997    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bn89p\" (UID: \"e31f7dc9-6732-44c5-a131-d9802a13613f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:04:59.285172 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:59.285072    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sswtl\" (UID: \"4f11e7df-616c-49a2-9071-0f32a5b25adf\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:04:59.285172 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:59.285115    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:04:59.285172 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.285152    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 22:04:59.285282 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.285213    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:04:59.285282 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.285224    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls podName:e31f7dc9-6732-44c5-a131-d9802a13613f nodeName:}" failed. No retries permitted until 2026-07-28 22:05:07.285204699 +0000 UTC m=+51.293070256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls") pod "cluster-samples-operator-6748489656-bn89p" (UID: "e31f7dc9-6732-44c5-a131-d9802a13613f") : secret "samples-operator-tls" not found
Jul 28 22:04:59.285282 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.285260    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls podName:8b67f675-10f2-4616-85c9-7feec90a410c nodeName:}" failed. No retries permitted until 2026-07-28 22:05:07.285249868 +0000 UTC m=+51.293115405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls") pod "dns-default-cqbpk" (UID: "8b67f675-10f2-4616-85c9-7feec90a410c") : secret "dns-default-metrics-tls" not found
Jul 28 22:04:59.285282 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.285214    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 22:04:59.285403 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:04:59.285287    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert podName:4f11e7df-616c-49a2-9071-0f32a5b25adf nodeName:}" failed. No retries permitted until 2026-07-28 22:05:07.285281776 +0000 UTC m=+51.293147312 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sswtl" (UID: "4f11e7df-616c-49a2-9071-0f32a5b25adf") : secret "networking-console-plugin-cert" not found
Jul 28 22:04:59.395710 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:59.395681    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:04:59.489075 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:04:59.489007    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:00.428382 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:00.428346    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:00.529842 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:00.529779    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:00.629499 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:00.629467    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:00.736235 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:00.736213    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:01.679545 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.679069    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-77hcw" event={"ID":"1465e125-b955-4c39-bc95-67661bd1f3a3","Type":"ContainerStarted","Data":"6315c58c75f240345a80b55d8bd549a7081e1e92f31b88cda0599205360f5730"}
Jul 28 22:05:01.680924 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.680895    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hjds4" event={"ID":"ac77877d-96a7-41ed-b29d-6e3dccfdecf3","Type":"ContainerStarted","Data":"7ccf42fac3180828d03ecdbbc5912ccc7331a9e78af58349f74a61298dd773f6"}
Jul 28 22:05:01.683072 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.683038    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl" event={"ID":"9d2c8128-4019-4802-8cb0-8307891fd7c5","Type":"ContainerStarted","Data":"abfc932f4fec36faea6edde4a40d9270f52baddaf93f7903d0bf938f134dc657"}
Jul 28 22:05:01.684766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.684745    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/0.log"
Jul 28 22:05:01.684896 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.684782    2583 generic.go:356] "Generic (PLEG): container finished" podID="095ffcbc-5ca5-483e-b0d5-c919b4b9ab88" containerID="e704279b4f26444df9840b075bf1a5373351ceec523e03992c45efb989f47dbd" exitCode=255
Jul 28 22:05:01.684896 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.684851    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4" event={"ID":"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88","Type":"ContainerDied","Data":"e704279b4f26444df9840b075bf1a5373351ceec523e03992c45efb989f47dbd"}
Jul 28 22:05:01.685103 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.685084    2583 scope.go:117] "RemoveContainer" containerID="e704279b4f26444df9840b075bf1a5373351ceec523e03992c45efb989f47dbd"
Jul 28 22:05:01.687687 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.687478    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4dj68" event={"ID":"145369c0-52c4-43e3-a1c3-154831f04b30","Type":"ContainerStarted","Data":"c55e51b8ddccb40efaa902ba6db614e4f693861f07ca429408fb7df80cf1bf43"}
Jul 28 22:05:01.689244 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.689224    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-th52b" event={"ID":"0fcee02e-7d41-4794-96bf-9d49280416c3","Type":"ContainerStarted","Data":"ced2a7af4e5347008e0f203ede2dbd875571f5559290dc96049961497f8d6111"}
Jul 28 22:05:01.690848 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.690777    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p" event={"ID":"3e9b3b23-0e89-4d3f-8f19-6830b0b68215","Type":"ContainerStarted","Data":"2a481e6f88fee366a5d0a481fabb030974c201420e2bae58fda2e2475030f0f0"}
Jul 28 22:05:01.694465 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.694442    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjwtm" event={"ID":"2a85189c-c2ee-4a08-bb3e-7ad54914bc63","Type":"ContainerStarted","Data":"456945ac1e09564efdae1c1687fba8fd7db2c0fb7deb46451282c872173a601a"}
Jul 28 22:05:01.698639 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.698588    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-77hcw" podStartSLOduration=16.889459006 podStartE2EDuration="24.69857116s" totalImagesPullingTime="7.809112154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:01.697228972 +0000 UTC m=+45.705094541" watchObservedRunningTime="2026-07-28 22:05:01.69857116 +0000 UTC m=+45.706436718"
Jul 28 22:05:01.733807 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.731854    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:05:01.733807 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.731916    2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:05:01.739623 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.739262    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-th52b" podStartSLOduration=16.95033739 podStartE2EDuration="24.739242925s" totalImagesPullingTime="7.788905535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:01.737709217 +0000 UTC m=+45.745574776" watchObservedRunningTime="2026-07-28 22:05:01.739242925 +0000 UTC m=+45.747108478"
Jul 28 22:05:01.755290 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.755251    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-4dj68" podStartSLOduration=16.707613995 podStartE2EDuration="24.755239289s" totalImagesPullingTime="8.047625294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:01.75505813 +0000 UTC m=+45.762923690" watchObservedRunningTime="2026-07-28 22:05:01.755239289 +0000 UTC m=+45.763104847"
Jul 28 22:05:01.789278 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.789228    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zjwtm" podStartSLOduration=14.352696592 podStartE2EDuration="44.789212318s" totalImagesPullingTime="30.436515726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:01.788030768 +0000 UTC m=+45.795896349" watchObservedRunningTime="2026-07-28 22:05:01.789212318 +0000 UTC m=+45.797077877"
Jul 28 22:05:01.824038 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.821214    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl" podStartSLOduration=16.987803251 podStartE2EDuration="24.821193052s" totalImagesPullingTime="7.833389801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:01.816408093 +0000 UTC m=+45.824273649" watchObservedRunningTime="2026-07-28 22:05:01.821193052 +0000 UTC m=+45.829058611"
Jul 28 22:05:01.824038 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.822713    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-npvph"]
Jul 28 22:05:01.843211 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.842223    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-npvph"]
Jul 28 22:05:01.843211 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.842362    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:01.846034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.845366    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 22:05:01.846034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.845874    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8gj46\""
Jul 28 22:05:01.847005 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.846920    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p" podStartSLOduration=17.043789366 podStartE2EDuration="24.846887605s" totalImagesPullingTime="7.803098239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:01.845649391 +0000 UTC m=+45.853514951" watchObservedRunningTime="2026-07-28 22:05:01.846887605 +0000 UTC m=+45.854753165"
Jul 28 22:05:01.847583 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.847566    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 22:05:01.864544 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.864432    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hjds4" podStartSLOduration=3.056599314 podStartE2EDuration="10.864414584s" totalImagesPullingTime="7.80781527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:01.862968241 +0000 UTC m=+45.870833800" watchObservedRunningTime="2026-07-28 22:05:01.864414584 +0000 UTC m=+45.872280145"
Jul 28 22:05:01.875766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.875702    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:01.910331 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.910223    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcwq7\" (UniqueName: \"kubernetes.io/projected/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-api-access-vcwq7\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:01.910483 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.910369    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/76b97389-7eb1-46ac-8aa9-cb86280b26db-crio-socket\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:01.910483 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.910422    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/76b97389-7eb1-46ac-8aa9-cb86280b26db-data-volume\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:01.910882 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.910521    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:01.910882 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:01.910641    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:02.000677 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.000642    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:02.011649 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.011617    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/76b97389-7eb1-46ac-8aa9-cb86280b26db-crio-socket\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:02.011819 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.011675    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/76b97389-7eb1-46ac-8aa9-cb86280b26db-data-volume\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:02.011819 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.011703    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:02.011819 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.011746    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/76b97389-7eb1-46ac-8aa9-cb86280b26db-crio-socket\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:02.011819 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.011811    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:02.012034 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:02.011836    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 22:05:02.012034 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:02.011901    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm podName:76b97389-7eb1-46ac-8aa9-cb86280b26db nodeName:}" failed. No retries permitted until 2026-07-28 22:05:02.511882119 +0000 UTC m=+46.519747663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-npvph" (UID: "76b97389-7eb1-46ac-8aa9-cb86280b26db") : configmap "kube-rbac-proxy" not found
Jul 28 22:05:02.012034 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:02.011908    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:02.012034 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:02.011955    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls podName:76b97389-7eb1-46ac-8aa9-cb86280b26db nodeName:}" failed. No retries permitted until 2026-07-28 22:05:02.511937925 +0000 UTC m=+46.519803461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-npvph" (UID: "76b97389-7eb1-46ac-8aa9-cb86280b26db") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:02.012034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.011973    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcwq7\" (UniqueName: \"kubernetes.io/projected/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-api-access-vcwq7\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:02.012287 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.012041    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/76b97389-7eb1-46ac-8aa9-cb86280b26db-data-volume\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:02.022900 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.022877    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcwq7\" (UniqueName: \"kubernetes.io/projected/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-api-access-vcwq7\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:02.100157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.100118    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:02.344574 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.344488    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:02.446624 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.446547    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:02.517546 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.517510    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:02.517740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.517577    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:02.517740 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:02.517640    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 22:05:02.517740 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:02.517705    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm podName:76b97389-7eb1-46ac-8aa9-cb86280b26db nodeName:}" failed. No retries permitted until 2026-07-28 22:05:03.517689519 +0000 UTC m=+47.525555055 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-npvph" (UID: "76b97389-7eb1-46ac-8aa9-cb86280b26db") : configmap "kube-rbac-proxy" not found
Jul 28 22:05:02.517935 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:02.517762    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:02.517935 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:02.517842    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls podName:76b97389-7eb1-46ac-8aa9-cb86280b26db nodeName:}" failed. No retries permitted until 2026-07-28 22:05:03.517824836 +0000 UTC m=+47.525690397 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-npvph" (UID: "76b97389-7eb1-46ac-8aa9-cb86280b26db") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:02.549048 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.549015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:02.644456 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.644428    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:02.701121 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.701085    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:05:02.701535 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.701501    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/0.log"
Jul 28 22:05:02.701596 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.701541    2583 generic.go:356] "Generic (PLEG): container finished" podID="095ffcbc-5ca5-483e-b0d5-c919b4b9ab88" containerID="c4320cb800e1869ae7c9da1d2d34174b3f434e79cc7b66bc4225fb9f2fb706af" exitCode=255
Jul 28 22:05:02.701665 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.701638    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4" event={"ID":"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88","Type":"ContainerDied","Data":"c4320cb800e1869ae7c9da1d2d34174b3f434e79cc7b66bc4225fb9f2fb706af"}
Jul 28 22:05:02.701729 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.701715    2583 scope.go:117] "RemoveContainer" containerID="e704279b4f26444df9840b075bf1a5373351ceec523e03992c45efb989f47dbd"
Jul 28 22:05:02.701928 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.701909    2583 scope.go:117] "RemoveContainer" containerID="c4320cb800e1869ae7c9da1d2d34174b3f434e79cc7b66bc4225fb9f2fb706af"
Jul 28 22:05:02.702123 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:02.702096    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-57xd4_openshift-console-operator(095ffcbc-5ca5-483e-b0d5-c919b4b9ab88)\"" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4" podUID="095ffcbc-5ca5-483e-b0d5-c919b4b9ab88"
Jul 28 22:05:02.774023 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.773990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:02.840292 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:02.840261    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:03.034840 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.034754    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lkmdh_769507e6-3089-4849-865d-283919d41c01/dns-node-resolver/0.log"
Jul 28 22:05:03.059101 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.059070    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:03.159935 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.159903    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:03.527200 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.527168    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:03.527366 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.527244    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:03.527366 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:03.527293    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 22:05:03.527366 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:03.527364    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:03.527459 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:03.527372    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm podName:76b97389-7eb1-46ac-8aa9-cb86280b26db nodeName:}" failed. No retries permitted until 2026-07-28 22:05:05.527355044 +0000 UTC m=+49.535220595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-npvph" (UID: "76b97389-7eb1-46ac-8aa9-cb86280b26db") : configmap "kube-rbac-proxy" not found
Jul 28 22:05:03.527459 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:03.527398    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls podName:76b97389-7eb1-46ac-8aa9-cb86280b26db nodeName:}" failed. No retries permitted until 2026-07-28 22:05:05.527388417 +0000 UTC m=+49.535253954 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-npvph" (UID: "76b97389-7eb1-46ac-8aa9-cb86280b26db") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:03.529341 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.529292    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-lnrxf"]
Jul 28 22:05:03.565962 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.565932    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:03.566437 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.566420    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-lnrxf"]
Jul 28 22:05:03.566526 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.566516    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lnrxf"
Jul 28 22:05:03.569117 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.569094    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 28 22:05:03.569753 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.569727    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 28 22:05:03.569871 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.569772    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-25jlz\""
Jul 28 22:05:03.627284 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.627250    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:03.627570 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.627554    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5bb6\" (UniqueName: \"kubernetes.io/projected/1fabc7ab-7d49-45c8-8757-00dbab295cdb-kube-api-access-b5bb6\") pod \"migrator-6865c9967-lnrxf\" (UID: \"1fabc7ab-7d49-45c8-8757-00dbab295cdb\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lnrxf"
Jul 28 22:05:03.706388 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.706358    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:05:03.706819 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.706715    2583 scope.go:117] "RemoveContainer" containerID="c4320cb800e1869ae7c9da1d2d34174b3f434e79cc7b66bc4225fb9f2fb706af"
Jul 28 22:05:03.706906 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:03.706887    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-57xd4_openshift-console-operator(095ffcbc-5ca5-483e-b0d5-c919b4b9ab88)\"" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4" podUID="095ffcbc-5ca5-483e-b0d5-c919b4b9ab88"
Jul 28 22:05:03.728468 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.728437    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bb6\" (UniqueName: \"kubernetes.io/projected/1fabc7ab-7d49-45c8-8757-00dbab295cdb-kube-api-access-b5bb6\") pod \"migrator-6865c9967-lnrxf\" (UID: \"1fabc7ab-7d49-45c8-8757-00dbab295cdb\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lnrxf"
Jul 28 22:05:03.735622 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.735595    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5bb6\" (UniqueName: \"kubernetes.io/projected/1fabc7ab-7d49-45c8-8757-00dbab295cdb-kube-api-access-b5bb6\") pod \"migrator-6865c9967-lnrxf\" (UID: \"1fabc7ab-7d49-45c8-8757-00dbab295cdb\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lnrxf"
Jul 28 22:05:03.875884 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:03.875840    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lnrxf"
Jul 28 22:05:04.019413 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.019380    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-lnrxf"]
Jul 28 22:05:04.022568 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:05:04.022537    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fabc7ab_7d49_45c8_8757_00dbab295cdb.slice/crio-dfe916101d3abcdebcc1e82a543bd74220345cd90b9283c725e48351972e0603 WatchSource:0}: Error finding container dfe916101d3abcdebcc1e82a543bd74220345cd90b9283c725e48351972e0603: Status 404 returned error can't find the container with id dfe916101d3abcdebcc1e82a543bd74220345cd90b9283c725e48351972e0603
Jul 28 22:05:04.221597 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.221505    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zvgg7_63b7aaeb-cc08-452a-89b7-59cf107dce99/node-ca/0.log"
Jul 28 22:05:04.441620 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.441586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:04.621371 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.621346    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hjds4_ac77877d-96a7-41ed-b29d-6e3dccfdecf3/serve-healthcheck-canary/0.log"
Jul 28 22:05:04.652712 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.652680    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-6ddvg"]
Jul 28 22:05:04.694364 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.694338    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-6ddvg"]
Jul 28 22:05:04.694533 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.694434    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg"
Jul 28 22:05:04.697149 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.697126    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 28 22:05:04.697264 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.697152    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 28 22:05:04.697264 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.697152    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 28 22:05:04.697858 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.697842    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-56lbt\""
Jul 28 22:05:04.697858 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.697852    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 28 22:05:04.703444 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.703415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:04.711297 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.711278    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lnrxf" event={"ID":"1fabc7ab-7d49-45c8-8757-00dbab295cdb","Type":"ContainerStarted","Data":"dfe916101d3abcdebcc1e82a543bd74220345cd90b9283c725e48351972e0603"}
Jul 28 22:05:04.737660 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.737640    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q8z5\" (UniqueName: \"kubernetes.io/projected/617a23fb-dcf2-4f26-b809-d9c3797a3a31-kube-api-access-5q8z5\") pod \"service-ca-658d7b7c78-6ddvg\" (UID: \"617a23fb-dcf2-4f26-b809-d9c3797a3a31\") " pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg"
Jul 28 22:05:04.737775 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.737685    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/617a23fb-dcf2-4f26-b809-d9c3797a3a31-signing-key\") pod \"service-ca-658d7b7c78-6ddvg\" (UID: \"617a23fb-dcf2-4f26-b809-d9c3797a3a31\") " pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg"
Jul 28 22:05:04.737775 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.737708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/617a23fb-dcf2-4f26-b809-d9c3797a3a31-signing-cabundle\") pod \"service-ca-658d7b7c78-6ddvg\" (UID: \"617a23fb-dcf2-4f26-b809-d9c3797a3a31\") " pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg"
Jul 28 22:05:04.775448 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.775415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:04.839331 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.839297    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q8z5\" (UniqueName: \"kubernetes.io/projected/617a23fb-dcf2-4f26-b809-d9c3797a3a31-kube-api-access-5q8z5\") pod \"service-ca-658d7b7c78-6ddvg\" (UID: \"617a23fb-dcf2-4f26-b809-d9c3797a3a31\") " pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg"
Jul 28 22:05:04.839501 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.839341    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/617a23fb-dcf2-4f26-b809-d9c3797a3a31-signing-key\") pod \"service-ca-658d7b7c78-6ddvg\" (UID: \"617a23fb-dcf2-4f26-b809-d9c3797a3a31\") " pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg"
Jul 28 22:05:04.839501 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.839366    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/617a23fb-dcf2-4f26-b809-d9c3797a3a31-signing-cabundle\") pod \"service-ca-658d7b7c78-6ddvg\" (UID: \"617a23fb-dcf2-4f26-b809-d9c3797a3a31\") " pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg"
Jul 28 22:05:04.840012 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.839992    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/617a23fb-dcf2-4f26-b809-d9c3797a3a31-signing-cabundle\") pod \"service-ca-658d7b7c78-6ddvg\" (UID: \"617a23fb-dcf2-4f26-b809-d9c3797a3a31\") " pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg"
Jul 28 22:05:04.841746 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.841727    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/617a23fb-dcf2-4f26-b809-d9c3797a3a31-signing-key\") pod \"service-ca-658d7b7c78-6ddvg\" (UID: \"617a23fb-dcf2-4f26-b809-d9c3797a3a31\") " pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg"
Jul 28 22:05:04.848215 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:04.848190    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q8z5\" (UniqueName: \"kubernetes.io/projected/617a23fb-dcf2-4f26-b809-d9c3797a3a31-kube-api-access-5q8z5\") pod \"service-ca-658d7b7c78-6ddvg\" (UID: \"617a23fb-dcf2-4f26-b809-d9c3797a3a31\") " pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg"
Jul 28 22:05:05.003447 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:05.003362    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg"
Jul 28 22:05:05.035075 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:05.035046    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:05.135750 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:05.135687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:05.159495 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:05.159463    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-6ddvg"]
Jul 28 22:05:05.162321 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:05:05.162293    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod617a23fb_dcf2_4f26_b809_d9c3797a3a31.slice/crio-78c8ff1f52758c5233e26e52bee1328084bbcb5c639ec516bcc50dea8a939e37 WatchSource:0}: Error finding container 78c8ff1f52758c5233e26e52bee1328084bbcb5c639ec516bcc50dea8a939e37: Status 404 returned error can't find the container with id 78c8ff1f52758c5233e26e52bee1328084bbcb5c639ec516bcc50dea8a939e37
Jul 28 22:05:05.545523 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:05.545433    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:05.545689 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:05.545610    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:05.545689 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:05.545629    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:05.545689 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:05.545684    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls podName:76b97389-7eb1-46ac-8aa9-cb86280b26db nodeName:}" failed. No retries permitted until 2026-07-28 22:05:09.545665807 +0000 UTC m=+53.553531358 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-npvph" (UID: "76b97389-7eb1-46ac-8aa9-cb86280b26db") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:05.545881 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:05.545731    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 22:05:05.545881 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:05.545777    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm podName:76b97389-7eb1-46ac-8aa9-cb86280b26db nodeName:}" failed. No retries permitted until 2026-07-28 22:05:09.545766383 +0000 UTC m=+53.553631919 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-npvph" (UID: "76b97389-7eb1-46ac-8aa9-cb86280b26db") : configmap "kube-rbac-proxy" not found
Jul 28 22:05:05.716304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:05.716267    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg" event={"ID":"617a23fb-dcf2-4f26-b809-d9c3797a3a31","Type":"ContainerStarted","Data":"f1ebf46e26687cba7f62c1d0dea6d66aecbcdce93734a97a4f6cc9b9f41f48f8"}
Jul 28 22:05:05.716684 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:05.716324    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg" event={"ID":"617a23fb-dcf2-4f26-b809-d9c3797a3a31","Type":"ContainerStarted","Data":"78c8ff1f52758c5233e26e52bee1328084bbcb5c639ec516bcc50dea8a939e37"}
Jul 28 22:05:05.726245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:05.726216    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:05.733147 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:05.733109    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-6ddvg" podStartSLOduration=1.733095364 podStartE2EDuration="1.733095364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:05:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:05.732396473 +0000 UTC m=+49.740262044" watchObservedRunningTime="2026-07-28 22:05:05.733095364 +0000 UTC m=+49.740960921"
Jul 28 22:05:06.215926 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:06.215893    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:06.319929 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:06.319839    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:06.488845 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:06.488774    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:06.673874 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:06.673840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:06.722165 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:06.722129    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lnrxf" event={"ID":"1fabc7ab-7d49-45c8-8757-00dbab295cdb","Type":"ContainerStarted","Data":"231563f8763f8db47289391bf76f7f0e2ee3944e383e6c828413ab02faba5c34"}
Jul 28 22:05:06.722556 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:06.722217    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lnrxf" event={"ID":"1fabc7ab-7d49-45c8-8757-00dbab295cdb","Type":"ContainerStarted","Data":"33d822b4c22b342867c4af0cbe484858e3d684df090c810b526a24fb8250fa7e"}
Jul 28 22:05:06.759583 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:06.759539    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:06.922204 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:06.922156    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:07.260747 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:07.260715    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:05:07.260984 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:07.260765    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:05:07.260984 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:07.260851    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:05:07.260984 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.260935    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle podName:5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e nodeName:}" failed. No retries permitted until 2026-07-28 22:05:23.260916729 +0000 UTC m=+67.268782265 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle") pod "router-default-6597fb4cff-sdft5" (UID: "5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e") : configmap references non-existent config key: service-ca.crt
Jul 28 22:05:07.260984 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.260966    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 22:05:07.261200 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.260985    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:05:07.261200 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.261000    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b9f75cc74-f72j8: secret "image-registry-tls" not found
Jul 28 22:05:07.261200 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.261020    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 22:05:07.261200 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.261035    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls podName:7bc1592a-3b91-40a0-979c-2799511bcbda nodeName:}" failed. No retries permitted until 2026-07-28 22:05:23.2610185 +0000 UTC m=+67.268884055 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-97gkk" (UID: "7bc1592a-3b91-40a0-979c-2799511bcbda") : secret "cluster-monitoring-operator-tls" not found
Jul 28 22:05:07.261200 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:07.260969    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:05:07.261200 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.261055    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls podName:96283224-d894-4e35-868c-028c1f572acf nodeName:}" failed. No retries permitted until 2026-07-28 22:05:23.261044042 +0000 UTC m=+67.268909583 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls") pod "image-registry-7b9f75cc74-f72j8" (UID: "96283224-d894-4e35-868c-028c1f572acf") : secret "image-registry-tls" not found
Jul 28 22:05:07.261200 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.261074    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs podName:5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e nodeName:}" failed. No retries permitted until 2026-07-28 22:05:23.261066346 +0000 UTC m=+67.268931887 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs") pod "router-default-6597fb4cff-sdft5" (UID: "5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e") : secret "router-metrics-certs-default" not found
Jul 28 22:05:07.306466 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:07.306432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:07.361876 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:07.361835    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:05:07.362031 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:07.361957    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bn89p\" (UID: \"e31f7dc9-6732-44c5-a131-d9802a13613f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:05:07.362031 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.361980    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:05:07.362154 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.362038    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls podName:8b67f675-10f2-4616-85c9-7feec90a410c nodeName:}" failed. No retries permitted until 2026-07-28 22:05:23.362022988 +0000 UTC m=+67.369888523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls") pod "dns-default-cqbpk" (UID: "8b67f675-10f2-4616-85c9-7feec90a410c") : secret "dns-default-metrics-tls" not found
Jul 28 22:05:07.362154 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:07.362068    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sswtl\" (UID: \"4f11e7df-616c-49a2-9071-0f32a5b25adf\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:05:07.362154 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.362088    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 22:05:07.362313 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.362162    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls podName:e31f7dc9-6732-44c5-a131-d9802a13613f nodeName:}" failed. No retries permitted until 2026-07-28 22:05:23.362146429 +0000 UTC m=+67.370011966 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls") pod "cluster-samples-operator-6748489656-bn89p" (UID: "e31f7dc9-6732-44c5-a131-d9802a13613f") : secret "samples-operator-tls" not found
Jul 28 22:05:07.362313 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.362211    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 22:05:07.362313 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:07.362284    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert podName:4f11e7df-616c-49a2-9071-0f32a5b25adf nodeName:}" failed. No retries permitted until 2026-07-28 22:05:23.362266536 +0000 UTC m=+67.370132077 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sswtl" (UID: "4f11e7df-616c-49a2-9071-0f32a5b25adf") : secret "networking-console-plugin-cert" not found
Jul 28 22:05:08.287961 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:08.287925    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:08.496059 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:08.496024    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:08.699779 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:08.699747    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:08.775849 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:08.775816    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:08.842015 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:08.841983    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:08.960155 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:08.960068    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:09.101103 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:09.101057    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:09.301405 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:09.301320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:09.581594 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:09.581497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:09.581930 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:09.581636    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:09.581930 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:09.581647    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:09.581930 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:09.581706    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls podName:76b97389-7eb1-46ac-8aa9-cb86280b26db nodeName:}" failed. No retries permitted until 2026-07-28 22:05:17.581691573 +0000 UTC m=+61.589557110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-npvph" (UID: "76b97389-7eb1-46ac-8aa9-cb86280b26db") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:05:09.582227 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:09.582206    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76b97389-7eb1-46ac-8aa9-cb86280b26db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:09.906625 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:09.906592    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:10.013853 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:10.013812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:10.302449 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:10.302360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:10.566589 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:10.566490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:10.728303 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:10.728244    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:10.806937 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:10.806898    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:10.900376 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:10.900343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:11.732055 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:11.732017    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:05:11.732445 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:11.732087    2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:05:11.732445 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:11.732426    2583 scope.go:117] "RemoveContainer" containerID="c4320cb800e1869ae7c9da1d2d34174b3f434e79cc7b66bc4225fb9f2fb706af"
Jul 28 22:05:11.732586 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:11.732567    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-57xd4_openshift-console-operator(095ffcbc-5ca5-483e-b0d5-c919b4b9ab88)\"" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4" podUID="095ffcbc-5ca5-483e-b0d5-c919b4b9ab88"
Jul 28 22:05:11.917675 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:11.917642    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:12.127388 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:12.127362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:12.429134 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:12.429057    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:12.531142 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:12.531108    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:12.691855 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:12.691758    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:12.895571 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:12.895535    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:14.096062 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:14.096020    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:14.294236 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:14.294204    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:14.517518 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:14.517483    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:14.696760 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:14.696728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:14.922657 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:14.922624    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:15.384579 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:15.384543    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:15.444833 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:15.444785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:15.607176 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:15.607148    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:15.708429 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:15.708352    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:15.892852 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:15.892821    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:16.091655 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:16.091563    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:16.492522 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:16.492487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:16.700003 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:16.699969    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:16.898329 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:16.898293    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:17.580267 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:17.580231    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:17.604943 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:17.604923    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hctzl"
Jul 28 22:05:17.638594 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:17.638549    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-lnrxf" podStartSLOduration=12.670306549 podStartE2EDuration="14.638533975s" totalImagesPullingTime="1.968227426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:05:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:06.744855911 +0000 UTC m=+50.752721489" watchObservedRunningTime="2026-07-28 22:05:17.638533975 +0000 UTC m=+61.646399586"
Jul 28 22:05:17.657533 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:17.657512    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:17.659631 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:17.659614    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76b97389-7eb1-46ac-8aa9-cb86280b26db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-npvph\" (UID: \"76b97389-7eb1-46ac-8aa9-cb86280b26db\") " pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:17.762653 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:17.762625    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8gj46\""
Jul 28 22:05:17.771098 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:17.771079    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-npvph"
Jul 28 22:05:17.898821 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:17.898779    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-npvph"]
Jul 28 22:05:17.902056 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:05:17.902031    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b97389_7eb1_46ac_8aa9_cb86280b26db.slice/crio-213484ed0178a5879685e4d451288e285a80c33bf46dc1fa212531aa627deb13 WatchSource:0}: Error finding container 213484ed0178a5879685e4d451288e285a80c33bf46dc1fa212531aa627deb13: Status 404 returned error can't find the container with id 213484ed0178a5879685e4d451288e285a80c33bf46dc1fa212531aa627deb13
Jul 28 22:05:18.764168 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:18.764068    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-npvph" event={"ID":"76b97389-7eb1-46ac-8aa9-cb86280b26db","Type":"ContainerStarted","Data":"213484ed0178a5879685e4d451288e285a80c33bf46dc1fa212531aa627deb13"}
Jul 28 22:05:22.098571 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.098526    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:05:22.102700 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.102666    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 22:05:22.111138 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.111111    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dae75d3b-5868-4cd2-b7e4-45f6a145b875-original-pull-secret\") pod \"global-pull-secret-syncer-t5bf9\" (UID: \"dae75d3b-5868-4cd2-b7e4-45f6a145b875\") " pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:05:22.137786 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.137754    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t5bf9"
Jul 28 22:05:22.199653 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.199622    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs\") pod \"network-metrics-daemon-5vh87\" (UID: \"33b4bdc7-694b-45e4-8c88-3cd9f134865d\") " pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:05:22.199861 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.199808    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk6vq\" (UniqueName: \"kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq\") pod \"network-check-target-btcz4\" (UID: \"b83ca49e-9d70-42ee-b256-2d4d995c598e\") " pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:05:22.202242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.202217    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk6vq\" (UniqueName: \"kubernetes.io/projected/b83ca49e-9d70-42ee-b256-2d4d995c598e-kube-api-access-kk6vq\") pod \"network-check-target-btcz4\" (UID: \"b83ca49e-9d70-42ee-b256-2d4d995c598e\") " pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:05:22.202465 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.202451    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 22:05:22.212334 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.212313    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33b4bdc7-694b-45e4-8c88-3cd9f134865d-metrics-certs\") pod \"network-metrics-daemon-5vh87\" (UID: \"33b4bdc7-694b-45e4-8c88-3cd9f134865d\") " pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:05:22.378509 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.378469    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bnqzv\""
Jul 28 22:05:22.386251 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.386216    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:05:22.426089 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.426056    2583 scope.go:117] "RemoveContainer" containerID="c4320cb800e1869ae7c9da1d2d34174b3f434e79cc7b66bc4225fb9f2fb706af"
Jul 28 22:05:22.446253 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.446212    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sm6xq\""
Jul 28 22:05:22.455041 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:22.455017    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5vh87"
Jul 28 22:05:23.139572 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.139397    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-btcz4"]
Jul 28 22:05:23.156166 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.156138    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5vh87"]
Jul 28 22:05:23.242919 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:05:23.242843    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb83ca49e_9d70_42ee_b256_2d4d995c598e.slice/crio-c3221b564fdfb519055792b659d0a1d23ea0694df5ddec98fee6aa80a624ebd8 WatchSource:0}: Error finding container c3221b564fdfb519055792b659d0a1d23ea0694df5ddec98fee6aa80a624ebd8: Status 404 returned error can't find the container with id c3221b564fdfb519055792b659d0a1d23ea0694df5ddec98fee6aa80a624ebd8
Jul 28 22:05:23.243956 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:05:23.243678    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b4bdc7_694b_45e4_8c88_3cd9f134865d.slice/crio-b86dba320ea0a60c8c2c6f5f9a6d9d9a5dd6da87c918c0a82d63018e9ee7e26e WatchSource:0}: Error finding container b86dba320ea0a60c8c2c6f5f9a6d9d9a5dd6da87c918c0a82d63018e9ee7e26e: Status 404 returned error can't find the container with id b86dba320ea0a60c8c2c6f5f9a6d9d9a5dd6da87c918c0a82d63018e9ee7e26e
Jul 28 22:05:23.310715 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.310692    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:05:23.310788 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.310748    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:05:23.310869 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.310787    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:05:23.310869 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.310836    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:05:23.311691 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.311640    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-service-ca-bundle\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:05:23.313298 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.313260    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls\") pod \"image-registry-7b9f75cc74-f72j8\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") " pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:05:23.313407 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.313369    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e-metrics-certs\") pod \"router-default-6597fb4cff-sdft5\" (UID: \"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e\") " pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:05:23.313501 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.313480    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc1592a-3b91-40a0-979c-2799511bcbda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-97gkk\" (UID: \"7bc1592a-3b91-40a0-979c-2799511bcbda\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:05:23.363171 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.362428    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t5bf9"]
Jul 28 22:05:23.366900 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:05:23.366861    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddae75d3b_5868_4cd2_b7e4_45f6a145b875.slice/crio-5508fb757e4e37a340ac81fb478f7d2bd533562bd76eeb61bffc34ce00056b31 WatchSource:0}: Error finding container 5508fb757e4e37a340ac81fb478f7d2bd533562bd76eeb61bffc34ce00056b31: Status 404 returned error can't find the container with id 5508fb757e4e37a340ac81fb478f7d2bd533562bd76eeb61bffc34ce00056b31
Jul 28 22:05:23.411491 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.411463    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sswtl\" (UID: \"4f11e7df-616c-49a2-9071-0f32a5b25adf\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:05:23.411600 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.411512    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:05:23.411600 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.411569    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bn89p\" (UID: \"e31f7dc9-6732-44c5-a131-d9802a13613f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:05:23.413702 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.413677    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b67f675-10f2-4616-85c9-7feec90a410c-metrics-tls\") pod \"dns-default-cqbpk\" (UID: \"8b67f675-10f2-4616-85c9-7feec90a410c\") " pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:05:23.413790 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.413715    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f11e7df-616c-49a2-9071-0f32a5b25adf-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sswtl\" (UID: \"4f11e7df-616c-49a2-9071-0f32a5b25adf\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:05:23.414058 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.414041    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e31f7dc9-6732-44c5-a131-d9802a13613f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bn89p\" (UID: \"e31f7dc9-6732-44c5-a131-d9802a13613f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:05:23.497664 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.497598    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2lgj6\""
Jul 28 22:05:23.506338 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.506319    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:05:23.508034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.508018    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-55ztt\""
Jul 28 22:05:23.516082 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.516065    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"
Jul 28 22:05:23.551789 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.551767    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qwjr6\""
Jul 28 22:05:23.560670 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.560603    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:05:23.564828 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.564744    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-cfr4m\""
Jul 28 22:05:23.571017 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.570772    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ch4rl\""
Jul 28 22:05:23.573672 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.573593    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"
Jul 28 22:05:23.579753 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.579514    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"
Jul 28 22:05:23.585056 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.584967    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8xbs4\""
Jul 28 22:05:23.593961 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.593848    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:05:23.741016 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.737941    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b9f75cc74-f72j8"]
Jul 28 22:05:23.741707 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:05:23.741325    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96283224_d894_4e35_868c_028c1f572acf.slice/crio-45e0eb427814eeda4f7fd41e9e63a30331565c88507f848a270fc6aa156618f4 WatchSource:0}: Error finding container 45e0eb427814eeda4f7fd41e9e63a30331565c88507f848a270fc6aa156618f4: Status 404 returned error can't find the container with id 45e0eb427814eeda4f7fd41e9e63a30331565c88507f848a270fc6aa156618f4
Jul 28 22:05:23.754989 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.754924    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk"]
Jul 28 22:05:23.759918 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:05:23.759895    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc1592a_3b91_40a0_979c_2799511bcbda.slice/crio-be90834725431cc8af15ce16a20e5278b3b9ec3644b2fd5a8e5d534a97efad6c WatchSource:0}: Error finding container be90834725431cc8af15ce16a20e5278b3b9ec3644b2fd5a8e5d534a97efad6c: Status 404 returned error can't find the container with id be90834725431cc8af15ce16a20e5278b3b9ec3644b2fd5a8e5d534a97efad6c
Jul 28 22:05:23.786591 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.786568    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:05:23.786693 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.786662    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4" event={"ID":"095ffcbc-5ca5-483e-b0d5-c919b4b9ab88","Type":"ContainerStarted","Data":"efe3bc06ddb16949c20f37594f103e0fd112124afc56ca453ad08f22f1e50eb8"}
Jul 28 22:05:23.787943 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.787876    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:05:23.789699 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.789633    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk" event={"ID":"7bc1592a-3b91-40a0-979c-2799511bcbda","Type":"ContainerStarted","Data":"be90834725431cc8af15ce16a20e5278b3b9ec3644b2fd5a8e5d534a97efad6c"}
Jul 28 22:05:23.792370 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.792278    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8" event={"ID":"96283224-d894-4e35-868c-028c1f572acf","Type":"ContainerStarted","Data":"45e0eb427814eeda4f7fd41e9e63a30331565c88507f848a270fc6aa156618f4"}
Jul 28 22:05:23.795225 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.794604    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-btcz4" event={"ID":"b83ca49e-9d70-42ee-b256-2d4d995c598e","Type":"ContainerStarted","Data":"1f0628b8b0815a80d9c5b09186282b8bb3f13a813977497f3ebd66315888241e"}
Jul 28 22:05:23.795225 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.794644    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-btcz4" event={"ID":"b83ca49e-9d70-42ee-b256-2d4d995c598e","Type":"ContainerStarted","Data":"c3221b564fdfb519055792b659d0a1d23ea0694df5ddec98fee6aa80a624ebd8"}
Jul 28 22:05:23.795225 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.795169    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:05:23.798238 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.798160    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5vh87" event={"ID":"33b4bdc7-694b-45e4-8c88-3cd9f134865d","Type":"ContainerStarted","Data":"b86dba320ea0a60c8c2c6f5f9a6d9d9a5dd6da87c918c0a82d63018e9ee7e26e"}
Jul 28 22:05:23.800071 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.800049    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-npvph" event={"ID":"76b97389-7eb1-46ac-8aa9-cb86280b26db","Type":"ContainerStarted","Data":"f39c127e657c445c7500a8f44f3807690568e17dac907a0bf5f9e57abe38e96d"}
Jul 28 22:05:23.801544 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.801521    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t5bf9" event={"ID":"dae75d3b-5868-4cd2-b7e4-45f6a145b875","Type":"ContainerStarted","Data":"5508fb757e4e37a340ac81fb478f7d2bd533562bd76eeb61bffc34ce00056b31"}
Jul 28 22:05:23.811948 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.811902    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4" podStartSLOduration=39.003815659 podStartE2EDuration="46.811871325s" totalImagesPullingTime="7.808055666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:23.807973868 +0000 UTC m=+67.815839426" watchObservedRunningTime="2026-07-28 22:05:23.811871325 +0000 UTC m=+67.819736880"
Jul 28 22:05:23.813727 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.813694    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-sswtl"]
Jul 28 22:05:23.825139 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.824849    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-btcz4" podStartSLOduration=66.824836368 podStartE2EDuration="1m6.824836368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:23.82424551 +0000 UTC m=+67.832111060" watchObservedRunningTime="2026-07-28 22:05:23.824836368 +0000 UTC m=+67.832701923"
Jul 28 22:05:23.837945 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.837917    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6597fb4cff-sdft5"]
Jul 28 22:05:23.840990 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:05:23.840963    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b4bd6f3_04c2_4a36_b0f7_f2ed0fc9914e.slice/crio-41607164d12548d3fced0cc27be1a5070ba31abaf5bdd9071d22e9a75515fcfa WatchSource:0}: Error finding container 41607164d12548d3fced0cc27be1a5070ba31abaf5bdd9071d22e9a75515fcfa: Status 404 returned error can't find the container with id 41607164d12548d3fced0cc27be1a5070ba31abaf5bdd9071d22e9a75515fcfa
Jul 28 22:05:23.917493 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:23.917464    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-57xd4"
Jul 28 22:05:24.070195 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.068600    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p"]
Jul 28 22:05:24.070195 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.068672    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cqbpk"]
Jul 28 22:05:24.129162 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.129089    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:24.231892 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.231836    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:24.331452 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.331360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:24.431469 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.431407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:24.681316 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.681284    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:24.809000 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.808962    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cqbpk" event={"ID":"8b67f675-10f2-4616-85c9-7feec90a410c","Type":"ContainerStarted","Data":"5df241ba0e5f1cfc0171735b5df15d98d4c2e9537e1d1d0522fee84314340ca1"}
Jul 28 22:05:24.811589 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.811553    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6597fb4cff-sdft5" event={"ID":"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e","Type":"ContainerStarted","Data":"f458399b2732341ad66d93e2201d5d550a2eb54cadb45acc9f684b9d89a5fac9"}
Jul 28 22:05:24.811731 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.811612    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6597fb4cff-sdft5" event={"ID":"5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e","Type":"ContainerStarted","Data":"41607164d12548d3fced0cc27be1a5070ba31abaf5bdd9071d22e9a75515fcfa"}
Jul 28 22:05:24.820183 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.819903    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8" event={"ID":"96283224-d894-4e35-868c-028c1f572acf","Type":"ContainerStarted","Data":"fb2375659d0f936db667de7f2fb590442b4c2d643e7fa709c320f5b7fead13c7"}
Jul 28 22:05:24.820496 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.820398    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:05:24.820496 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.820447    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:24.821846 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.821767    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p" event={"ID":"e31f7dc9-6732-44c5-a131-d9802a13613f","Type":"ContainerStarted","Data":"37030d3dc0cffad30e57e4eec722c68e040ef6d27e3477f65a21a3e10cdb36ce"}
Jul 28 22:05:24.824314 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.824291    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl" event={"ID":"4f11e7df-616c-49a2-9071-0f32a5b25adf","Type":"ContainerStarted","Data":"1c16eb9c213054d6374225ae00dfaf124e51eff786ae5782311c608d6d8904c9"}
Jul 28 22:05:24.873216 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.873158    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8" podStartSLOduration=67.873134988 podStartE2EDuration="1m7.873134988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:24.872543132 +0000 UTC m=+68.880408691" watchObservedRunningTime="2026-07-28 22:05:24.873134988 +0000 UTC m=+68.881000547"
Jul 28 22:05:24.873399 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.873363    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6597fb4cff-sdft5" podStartSLOduration=47.873356415 podStartE2EDuration="47.873356415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:24.851228176 +0000 UTC m=+68.859093752" watchObservedRunningTime="2026-07-28 22:05:24.873356415 +0000 UTC m=+68.881221974"
Jul 28 22:05:24.925353 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:24.925248    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:25.028010 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.027892    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:25.201415 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.201377    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:25.212752 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.212695    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-llh92"]
Jul 28 22:05:25.234220 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.233159    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-llh92"
Jul 28 22:05:25.236551 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.235941    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 22:05:25.236551 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.236365    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gc5dt\""
Jul 28 22:05:25.239403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.239228    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 22:05:25.258311 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.258289    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-llh92"]
Jul 28 22:05:25.330957 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.330924    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:25.334974 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.334905    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gttpg\" (UniqueName: \"kubernetes.io/projected/08aa7923-c078-4a4f-9577-55ccac85fe3a-kube-api-access-gttpg\") pod \"downloads-7c97dc4555-llh92\" (UID: \"08aa7923-c078-4a4f-9577-55ccac85fe3a\") " pod="openshift-console/downloads-7c97dc4555-llh92"
Jul 28 22:05:25.435871 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.435530    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gttpg\" (UniqueName: \"kubernetes.io/projected/08aa7923-c078-4a4f-9577-55ccac85fe3a-kube-api-access-gttpg\") pod \"downloads-7c97dc4555-llh92\" (UID: \"08aa7923-c078-4a4f-9577-55ccac85fe3a\") " pod="openshift-console/downloads-7c97dc4555-llh92"
Jul 28 22:05:25.462410 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.462326    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gttpg\" (UniqueName: \"kubernetes.io/projected/08aa7923-c078-4a4f-9577-55ccac85fe3a-kube-api-access-gttpg\") pod \"downloads-7c97dc4555-llh92\" (UID: \"08aa7923-c078-4a4f-9577-55ccac85fe3a\") " pod="openshift-console/downloads-7c97dc4555-llh92"
Jul 28 22:05:25.527390 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.527255    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:25.559964 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.559677    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-llh92"
Jul 28 22:05:25.561657 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.561640    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:05:25.567403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.567215    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:05:25.632177 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.632060    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:25.772042 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.772005    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-llh92"]
Jul 28 22:05:25.777481 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:05:25.777451    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08aa7923_c078_4a4f_9577_55ccac85fe3a.slice/crio-fcd16553e0d448f5997ae03e0bbb02c134abac8dc33f5e1e80496b8d8e8fc71a WatchSource:0}: Error finding container fcd16553e0d448f5997ae03e0bbb02c134abac8dc33f5e1e80496b8d8e8fc71a: Status 404 returned error can't find the container with id fcd16553e0d448f5997ae03e0bbb02c134abac8dc33f5e1e80496b8d8e8fc71a
Jul 28 22:05:25.835651 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.835530    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-llh92" event={"ID":"08aa7923-c078-4a4f-9577-55ccac85fe3a","Type":"ContainerStarted","Data":"fcd16553e0d448f5997ae03e0bbb02c134abac8dc33f5e1e80496b8d8e8fc71a"}
Jul 28 22:05:25.841232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.841034    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5vh87" event={"ID":"33b4bdc7-694b-45e4-8c88-3cd9f134865d","Type":"ContainerStarted","Data":"d986cdae579cf589594c77768dc96037303db703c083f85aac19f8b855346008"}
Jul 28 22:05:25.841232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.841100    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5vh87" event={"ID":"33b4bdc7-694b-45e4-8c88-3cd9f134865d","Type":"ContainerStarted","Data":"cac1b57b1481ddb77ea4343d43c7d8db0a49db99aacf8bd9fa1bd219a74cfa18"}
Jul 28 22:05:25.842440 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.842366    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:05:25.844271 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.844227    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6597fb4cff-sdft5"
Jul 28 22:05:25.862923 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:25.862009    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5vh87" podStartSLOduration=66.986453075 podStartE2EDuration="1m8.861990471s" totalImagesPullingTime="1.875537396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:25.860561878 +0000 UTC m=+69.868427461" watchObservedRunningTime="2026-07-28 22:05:25.861990471 +0000 UTC m=+69.869856031"
Jul 28 22:05:26.085232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:26.084466    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:26.530828 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:26.530728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:26.943790 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:26.943747    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:27.044472 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:27.044432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:27.129859 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:27.129828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:27.343803 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:27.343653    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:27.401343 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:27.401304    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:27.814332 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:27.814225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:28.102305 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:28.102147    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:30.443593 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:30.443524    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:30.500358 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:30.500272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:30.605659 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:30.605610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:30.664234 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:30.664168    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:30.888587 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:30.888542    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:31.000143 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:31.000040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:32.667839 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:32.667723    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:32.834965 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:32.834917    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:32.875814 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:32.875749    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t5bf9" event={"ID":"dae75d3b-5868-4cd2-b7e4-45f6a145b875","Type":"ContainerStarted","Data":"ae6f5dba0020477522326572bad31df5b60c0959730215bd4e1450648b176045"}
Jul 28 22:05:32.877855 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:32.877827    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cqbpk" event={"ID":"8b67f675-10f2-4616-85c9-7feec90a410c","Type":"ContainerStarted","Data":"8af5ec68920eb824a3a7ae7178bd1549da2479ca1dea76219a8b7e1b00df10a6"}
Jul 28 22:05:32.879491 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:32.879469    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk" event={"ID":"7bc1592a-3b91-40a0-979c-2799511bcbda","Type":"ContainerStarted","Data":"db64b7f961ae04bb6e704d51be8c9dda29825600a7c397ff5aaf2cf4194fe30e"}
Jul 28 22:05:32.881613 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:32.881588    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p" event={"ID":"e31f7dc9-6732-44c5-a131-d9802a13613f","Type":"ContainerStarted","Data":"6726d50f0a23933fffc91d130088d4083abb5d92d82ed2fa8850daf14e93643d"}
Jul 28 22:05:32.883550 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:32.883528    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl" event={"ID":"4f11e7df-616c-49a2-9071-0f32a5b25adf","Type":"ContainerStarted","Data":"98ed6a75f2eeb95fda08232ca3bb0587731f0982052a6ab1a93934865ecf5909"}
Jul 28 22:05:32.897908 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:32.897844    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t5bf9" podStartSLOduration=66.697973494 podStartE2EDuration="1m15.897825667s" totalImagesPullingTime="9.199852173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:32.896079585 +0000 UTC m=+76.903945159" watchObservedRunningTime="2026-07-28 22:05:32.897825667 +0000 UTC m=+76.905691224"
Jul 28 22:05:32.918521 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:32.918467    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-97gkk" podStartSLOduration=47.506829179 podStartE2EDuration="55.918448441s" totalImagesPullingTime="8.411619262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:32.91682236 +0000 UTC m=+76.924687929" watchObservedRunningTime="2026-07-28 22:05:32.918448441 +0000 UTC m=+76.926314002"
Jul 28 22:05:32.940722 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:32.940455    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sswtl" podStartSLOduration=47.209657031 podStartE2EDuration="55.940436136s" totalImagesPullingTime="8.730779105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:32.938962716 +0000 UTC m=+76.946828275" watchObservedRunningTime="2026-07-28 22:05:32.940436136 +0000 UTC m=+76.948301693"
Jul 28 22:05:33.090019 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:33.089739    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:33.180599 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:33.180550    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:33.279343 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:33.279237    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:33.527675 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:33.527640    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:33.890893 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:33.890858    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-npvph" event={"ID":"76b97389-7eb1-46ac-8aa9-cb86280b26db","Type":"ContainerStarted","Data":"8821911f2af36a999a5e446718b0e923fb1d837db5a014b146afce5c56e7f95e"}
Jul 28 22:05:33.892845 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:33.892778    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cqbpk" event={"ID":"8b67f675-10f2-4616-85c9-7feec90a410c","Type":"ContainerStarted","Data":"8170d7b719d87de5c813b41b83262d62ac6f8994b7b191bf833c56ce3e2653ef"}
Jul 28 22:05:33.892983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:33.892891    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:05:33.894751 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:33.894726    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p" event={"ID":"e31f7dc9-6732-44c5-a131-d9802a13613f","Type":"ContainerStarted","Data":"63ae0066d6447e255e6318b40864548ee0fc9cc5b6613f6f9e73a771d37237d0"}
Jul 28 22:05:33.909808 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:33.909746    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cqbpk" podStartSLOduration=34.433477855 podStartE2EDuration="42.909727654s" totalImagesPullingTime="8.476249799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:33.908285981 +0000 UTC m=+77.916151569" watchObservedRunningTime="2026-07-28 22:05:33.909727654 +0000 UTC m=+77.917593210"
Jul 28 22:05:33.923977 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:33.923928    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bn89p" podStartSLOduration=48.557503858 podStartE2EDuration="56.923910873s" totalImagesPullingTime="8.366407015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:33.922924276 +0000 UTC m=+77.930789869" watchObservedRunningTime="2026-07-28 22:05:33.923910873 +0000 UTC m=+77.931776430"
Jul 28 22:05:35.905647 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:35.905607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:36.138284 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:36.138238    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:36.265648 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:36.265532    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:37.450859 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:37.450755    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:37.554034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:37.554007    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:37.776247 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:37.776160    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:37.877430 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:37.877389    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:37.916709 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:37.916675    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-npvph" event={"ID":"76b97389-7eb1-46ac-8aa9-cb86280b26db","Type":"ContainerStarted","Data":"12d6e0d1421d1370b158fec010706743b7e28ed259bf327dfa759451432dca6c"}
Jul 28 22:05:37.937608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:37.937552    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-npvph" podStartSLOduration=17.637475628 podStartE2EDuration="36.937536164s" totalImagesPullingTime="19.300060536s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:05:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:37.935742192 +0000 UTC m=+81.943607798" watchObservedRunningTime="2026-07-28 22:05:37.937536164 +0000 UTC m=+81.945401722"
Jul 28 22:05:37.945039 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:37.944977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:38.231393 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:38.231260    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:38.332374 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:38.332336    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:40.386653 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:40.386618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:40.505888 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:40.505848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:40.610262 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:40.610226    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:42.038311 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:42.038266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:42.139953 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:42.139915    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:43.903203 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:43.903176    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cqbpk"
Jul 28 22:05:44.581484 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:44.581444    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:44.682124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:44.682056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:45.172114 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:45.172078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:45.272345 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:45.272292    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:45.594167 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:45.594078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:45.694568 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:45.694526    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:45.846292 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:45.846224    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:05:46.139520 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:46.139487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:46.243391 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:46.243330    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:47.794500 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:47.794462    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:47.839259 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:47.839220    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b9f75cc74-f72j8"]
Jul 28 22:05:47.919526 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:47.919492    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:47.961051 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:47.961001    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-llh92" event={"ID":"08aa7923-c078-4a4f-9577-55ccac85fe3a","Type":"ContainerStarted","Data":"39837b9bc7d1bb075f5d29cc27710f11f61ca1b552532b9942c736eed53515eb"}
Jul 28 22:05:47.961220 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:47.961169    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-llh92"
Jul 28 22:05:47.973744 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:47.973714    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-llh92"
Jul 28 22:05:47.977783 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:47.977731    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-llh92" podStartSLOduration=1.715084514 podStartE2EDuration="22.977711163s" totalImagesPullingTime="21.262626649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:47.976756957 +0000 UTC m=+91.984622540" watchObservedRunningTime="2026-07-28 22:05:47.977711163 +0000 UTC m=+91.985576722"
Jul 28 22:05:48.188123 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:48.188084    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:48.242918 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:48.242877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:50.370205 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.370172    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:50.587705 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.587675    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:50.640569 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.640489    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mrpm6"]
Jul 28 22:05:50.674013 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.673981    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.681157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.678601    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 22:05:50.681157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.678859    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 22:05:50.681995 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.681753    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 22:05:50.682413 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.682093    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gsb4z\""
Jul 28 22:05:50.684656 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.683194    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 22:05:50.695686 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.695623    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:50.761065 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.761023    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-textfile\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.761237 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.761078    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xfdb\" (UniqueName: \"kubernetes.io/projected/6401cb70-70af-4012-acfc-a753e352f84d-kube-api-access-2xfdb\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.761237 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.761123    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.761237 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.761159    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6401cb70-70af-4012-acfc-a753e352f84d-metrics-client-ca\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.761237 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.761200    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-wtmp\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.761237 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.761229    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6401cb70-70af-4012-acfc-a753e352f84d-sys\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.761476 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.761268    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-tls\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.761476 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.761302    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.761476 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.761330    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6401cb70-70af-4012-acfc-a753e352f84d-root\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.803046 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.803016    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:50.862494 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.862449    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-textfile\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.862684 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.862501    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xfdb\" (UniqueName: \"kubernetes.io/projected/6401cb70-70af-4012-acfc-a753e352f84d-kube-api-access-2xfdb\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.862940 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.862918    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.863128 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.863110    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6401cb70-70af-4012-acfc-a753e352f84d-metrics-client-ca\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.863316 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.862953    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-textfile\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.863408 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.863263    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-wtmp\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.863564 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.863493    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-wtmp\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.863564 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.863509    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6401cb70-70af-4012-acfc-a753e352f84d-sys\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.863691 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.863587    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-tls\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.863691 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.863628    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.863691 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.863661    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6401cb70-70af-4012-acfc-a753e352f84d-root\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.863691 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.863669    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6401cb70-70af-4012-acfc-a753e352f84d-metrics-client-ca\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.863919 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.863741    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6401cb70-70af-4012-acfc-a753e352f84d-root\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.863919 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:50.863852    2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 22:05:50.864022 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:05:50.863922    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-tls podName:6401cb70-70af-4012-acfc-a753e352f84d nodeName:}" failed. No retries permitted until 2026-07-28 22:05:51.363901605 +0000 UTC m=+95.371767145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-tls") pod "node-exporter-mrpm6" (UID: "6401cb70-70af-4012-acfc-a753e352f84d") : secret "node-exporter-tls" not found
Jul 28 22:05:50.864196 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.864174    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6401cb70-70af-4012-acfc-a753e352f84d-sys\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.864303 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.864227    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.865564 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.865544    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.877867 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.877846    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xfdb\" (UniqueName: \"kubernetes.io/projected/6401cb70-70af-4012-acfc-a753e352f84d-kube-api-access-2xfdb\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:50.905641 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:50.905575    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:51.007366 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:51.007270    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:51.107838 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:51.107775    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:51.306826 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:51.306718    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:51.369049 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:51.368972    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-tls\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:51.371544 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:51.371520    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6401cb70-70af-4012-acfc-a753e352f84d-node-exporter-tls\") pod \"node-exporter-mrpm6\" (UID: \"6401cb70-70af-4012-acfc-a753e352f84d\") " pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:51.408388 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:51.408362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:51.596181 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:51.596108    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mrpm6"
Jul 28 22:05:51.611308 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:51.611274    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:51.632528 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:05:51.632474    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6401cb70_70af_4012_acfc_a753e352f84d.slice/crio-f4da7cdb23cb5ab0d044eefc9381fd5d097df49cc161b93251f0728e3b081909 WatchSource:0}: Error finding container f4da7cdb23cb5ab0d044eefc9381fd5d097df49cc161b93251f0728e3b081909: Status 404 returned error can't find the container with id f4da7cdb23cb5ab0d044eefc9381fd5d097df49cc161b93251f0728e3b081909
Jul 28 22:05:51.763418 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:51.763380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:51.981488 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:51.981453    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mrpm6" event={"ID":"6401cb70-70af-4012-acfc-a753e352f84d","Type":"ContainerStarted","Data":"f4da7cdb23cb5ab0d044eefc9381fd5d097df49cc161b93251f0728e3b081909"}
Jul 28 22:05:52.263069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:52.262978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:52.530857 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:52.530752    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:52.603573 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:52.603522    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:53.619176 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:53.619141    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:53.718763 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:53.718683    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:53.929856 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:53.929741    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:53.994046 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:53.994006    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mrpm6" event={"ID":"6401cb70-70af-4012-acfc-a753e352f84d","Type":"ContainerStarted","Data":"ec4fe914a95c1bcd91f8fc3f62a71b96e311c3d48a9d933a7e751b2024790245"}
Jul 28 22:05:54.167934 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:54.167855    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:54.265356 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:54.265319    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:54.327429 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:54.327391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:55.000145 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:55.000109    2583 generic.go:356] "Generic (PLEG): container finished" podID="6401cb70-70af-4012-acfc-a753e352f84d" containerID="ec4fe914a95c1bcd91f8fc3f62a71b96e311c3d48a9d933a7e751b2024790245" exitCode=0
Jul 28 22:05:55.000553 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:55.000194    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mrpm6" event={"ID":"6401cb70-70af-4012-acfc-a753e352f84d","Type":"ContainerDied","Data":"ec4fe914a95c1bcd91f8fc3f62a71b96e311c3d48a9d933a7e751b2024790245"}
Jul 28 22:05:55.137869 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:55.137835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:55.239272 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:55.239239    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:55.399894 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:55.399847    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:55.503908 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:55.503871    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:55.706132 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:55.706096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:55.845143 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:55.845108    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-btcz4"
Jul 28 22:05:56.007112 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:56.007020    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mrpm6" event={"ID":"6401cb70-70af-4012-acfc-a753e352f84d","Type":"ContainerStarted","Data":"a6e8fe4ce9bebd22e0b549a830a67a819765002e140d19ba7f1145e19851dc48"}
Jul 28 22:05:56.007550 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:56.007126    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mrpm6" event={"ID":"6401cb70-70af-4012-acfc-a753e352f84d","Type":"ContainerStarted","Data":"9c83e991a31be09c13b67c1d97595842956ec728b14db482a72e388f33446d5b"}
Jul 28 22:05:56.024567 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:56.024500    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mrpm6" podStartSLOduration=3.735446401 podStartE2EDuration="6.024485805s" totalImagesPullingTime="2.193483698s" totalInitContainerRuntime="95.555706ms" isStatefulPod=true podCreationTimestamp="2026-07-28 22:05:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:05:56.02384236 +0000 UTC m=+100.031707944" watchObservedRunningTime="2026-07-28 22:05:56.024485805 +0000 UTC m=+100.032351397"
Jul 28 22:05:56.914743 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:56.914708    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:57.277434 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.277351    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fd7df6c4b-ctctz"]
Jul 28 22:05:57.335935 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.335908    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fd7df6c4b-ctctz"]
Jul 28 22:05:57.336083 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.335999    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.338317 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.338291    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 22:05:57.338445 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.338339    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 22:05:57.338445 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.338349    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2k6rq\""
Jul 28 22:05:57.338537 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.338469    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 22:05:57.338868 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.338853    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 22:05:57.338947 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.338909    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 22:05:57.342791 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.342772    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 22:05:57.427157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.427130    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e957a432-274a-4a51-b300-032985eef46a-console-serving-cert\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.427290 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.427183    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-trusted-ca-bundle\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.427290 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.427256    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-oauth-serving-cert\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.427414 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.427293    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-service-ca\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.427414 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.427321    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmmsl\" (UniqueName: \"kubernetes.io/projected/e957a432-274a-4a51-b300-032985eef46a-kube-api-access-tmmsl\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.427414 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.427348    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e957a432-274a-4a51-b300-032985eef46a-console-oauth-config\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.427414 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.427390    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-console-config\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.474111 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.474087    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:57.528270 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.528208    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e957a432-274a-4a51-b300-032985eef46a-console-serving-cert\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.528270 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.528264    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-trusted-ca-bundle\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.528394 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.528290    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-oauth-serving-cert\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.528394 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.528311    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-service-ca\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.528394 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.528332    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmmsl\" (UniqueName: \"kubernetes.io/projected/e957a432-274a-4a51-b300-032985eef46a-kube-api-access-tmmsl\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.528506 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.528460    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e957a432-274a-4a51-b300-032985eef46a-console-oauth-config\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.528506 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.528493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-console-config\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.529083 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.529061    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-service-ca\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.529180 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.529158    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-oauth-serving-cert\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.529226 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.529181    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-console-config\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.529529 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.529511    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-trusted-ca-bundle\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.530887 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.530859    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e957a432-274a-4a51-b300-032985eef46a-console-oauth-config\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.530887 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.530873    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e957a432-274a-4a51-b300-032985eef46a-console-serving-cert\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.535354 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.535333    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmmsl\" (UniqueName: \"kubernetes.io/projected/e957a432-274a-4a51-b300-032985eef46a-kube-api-access-tmmsl\") pod \"console-6fd7df6c4b-ctctz\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") " pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.575151 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.575126    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:05:57.646006 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.645980    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:05:57.801278 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:57.801139    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fd7df6c4b-ctctz"]
Jul 28 22:05:57.803895 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:05:57.803862    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode957a432_274a_4a51_b300_032985eef46a.slice/crio-4c48800dc550b22b65a7beed760d125d581287044a54c60c5bd02de6dda4f3ee WatchSource:0}: Error finding container 4c48800dc550b22b65a7beed760d125d581287044a54c60c5bd02de6dda4f3ee: Status 404 returned error can't find the container with id 4c48800dc550b22b65a7beed760d125d581287044a54c60c5bd02de6dda4f3ee
Jul 28 22:05:58.016108 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:05:58.016069    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fd7df6c4b-ctctz" event={"ID":"e957a432-274a-4a51-b300-032985eef46a","Type":"ContainerStarted","Data":"4c48800dc550b22b65a7beed760d125d581287044a54c60c5bd02de6dda4f3ee"}
Jul 28 22:06:00.601714 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:00.601652    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:06:00.661190 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:00.661153    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:06:02.034975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:02.034940    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fd7df6c4b-ctctz" event={"ID":"e957a432-274a-4a51-b300-032985eef46a","Type":"ContainerStarted","Data":"b442016faa64b330be1cab5e54e8efb5e0bb1e415d921f5f6a31a97edc580e4a"}
Jul 28 22:06:02.051965 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:02.051911    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fd7df6c4b-ctctz" podStartSLOduration=1.755232862 podStartE2EDuration="5.051895167s" totalImagesPullingTime="3.296662305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:05:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:06:02.050144252 +0000 UTC m=+106.058009811" watchObservedRunningTime="2026-07-28 22:06:02.051895167 +0000 UTC m=+106.059760725"
Jul 28 22:06:07.054692 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:07.054654    2583 generic.go:356] "Generic (PLEG): container finished" podID="9d2c8128-4019-4802-8cb0-8307891fd7c5" containerID="abfc932f4fec36faea6edde4a40d9270f52baddaf93f7903d0bf938f134dc657" exitCode=0
Jul 28 22:06:07.055359 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:07.054710    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl" event={"ID":"9d2c8128-4019-4802-8cb0-8307891fd7c5","Type":"ContainerDied","Data":"abfc932f4fec36faea6edde4a40d9270f52baddaf93f7903d0bf938f134dc657"}
Jul 28 22:06:07.055591 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:07.055572    2583 scope.go:117] "RemoveContainer" containerID="abfc932f4fec36faea6edde4a40d9270f52baddaf93f7903d0bf938f134dc657"
Jul 28 22:06:07.646568 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:07.646530    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:06:07.646732 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:07.646608    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:06:07.651750 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:07.651692    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:06:08.060643 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:08.060563    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-kb5jl" event={"ID":"9d2c8128-4019-4802-8cb0-8307891fd7c5","Type":"ContainerStarted","Data":"f6bf837d785ec1ec64538c4e52e38152ee22f7ec6a29e54d2d71928620f2209d"}
Jul 28 22:06:08.064263 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:08.064242    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:06:08.233048 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:08.233014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:06:08.330351 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:08.330272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:06:08.385608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:08.385570    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:06:12.076704 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:12.076677    2583 generic.go:356] "Generic (PLEG): container finished" podID="3e9b3b23-0e89-4d3f-8f19-6830b0b68215" containerID="2a481e6f88fee366a5d0a481fabb030974c201420e2bae58fda2e2475030f0f0" exitCode=0
Jul 28 22:06:12.077049 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:12.076730    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p" event={"ID":"3e9b3b23-0e89-4d3f-8f19-6830b0b68215","Type":"ContainerDied","Data":"2a481e6f88fee366a5d0a481fabb030974c201420e2bae58fda2e2475030f0f0"}
Jul 28 22:06:12.077049 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:12.077020    2583 scope.go:117] "RemoveContainer" containerID="2a481e6f88fee366a5d0a481fabb030974c201420e2bae58fda2e2475030f0f0"
Jul 28 22:06:12.862621 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:12.862579    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8" podUID="96283224-d894-4e35-868c-028c1f572acf" containerName="registry" containerID="cri-o://fb2375659d0f936db667de7f2fb590442b4c2d643e7fa709c320f5b7fead13c7" gracePeriod=30
Jul 28 22:06:13.083947 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.083914    2583 generic.go:356] "Generic (PLEG): container finished" podID="96283224-d894-4e35-868c-028c1f572acf" containerID="fb2375659d0f936db667de7f2fb590442b4c2d643e7fa709c320f5b7fead13c7" exitCode=0
Jul 28 22:06:13.084354 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.083986    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8" event={"ID":"96283224-d894-4e35-868c-028c1f572acf","Type":"ContainerDied","Data":"fb2375659d0f936db667de7f2fb590442b4c2d643e7fa709c320f5b7fead13c7"}
Jul 28 22:06:13.085928 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.085904    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p" event={"ID":"3e9b3b23-0e89-4d3f-8f19-6830b0b68215","Type":"ContainerStarted","Data":"764aa874672b5d92b5be56f147cc4771e06e8df2b35e4bc9874d37babeb4811a"}
Jul 28 22:06:13.138330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.138309    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:06:13.260407 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.260375    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/96283224-d894-4e35-868c-028c1f572acf-trusted-ca\") pod \"96283224-d894-4e35-868c-028c1f572acf\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") "
Jul 28 22:06:13.260608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.260437    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/96283224-d894-4e35-868c-028c1f572acf-installation-pull-secrets\") pod \"96283224-d894-4e35-868c-028c1f572acf\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") "
Jul 28 22:06:13.260608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.260465    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-bound-sa-token\") pod \"96283224-d894-4e35-868c-028c1f572acf\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") "
Jul 28 22:06:13.260608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.260510    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/96283224-d894-4e35-868c-028c1f572acf-image-registry-private-configuration\") pod \"96283224-d894-4e35-868c-028c1f572acf\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") "
Jul 28 22:06:13.260608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.260536    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/96283224-d894-4e35-868c-028c1f572acf-ca-trust-extracted\") pod \"96283224-d894-4e35-868c-028c1f572acf\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") "
Jul 28 22:06:13.260608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.260560    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bc8g7\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-kube-api-access-bc8g7\") pod \"96283224-d894-4e35-868c-028c1f572acf\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") "
Jul 28 22:06:13.260608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.260588    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/96283224-d894-4e35-868c-028c1f572acf-registry-certificates\") pod \"96283224-d894-4e35-868c-028c1f572acf\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") "
Jul 28 22:06:13.260932 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.260642    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls\") pod \"96283224-d894-4e35-868c-028c1f572acf\" (UID: \"96283224-d894-4e35-868c-028c1f572acf\") "
Jul 28 22:06:13.260932 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.260861    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96283224-d894-4e35-868c-028c1f572acf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "96283224-d894-4e35-868c-028c1f572acf" (UID: "96283224-d894-4e35-868c-028c1f572acf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:06:13.261029 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.261018    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/96283224-d894-4e35-868c-028c1f572acf-trusted-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:06:13.261192 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.261165    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96283224-d894-4e35-868c-028c1f572acf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "96283224-d894-4e35-868c-028c1f572acf" (UID: "96283224-d894-4e35-868c-028c1f572acf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:06:13.263057 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.263024    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96283224-d894-4e35-868c-028c1f572acf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "96283224-d894-4e35-868c-028c1f572acf" (UID: "96283224-d894-4e35-868c-028c1f572acf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:06:13.263178 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.263070    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "96283224-d894-4e35-868c-028c1f572acf" (UID: "96283224-d894-4e35-868c-028c1f572acf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:06:13.263178 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.263121    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "96283224-d894-4e35-868c-028c1f572acf" (UID: "96283224-d894-4e35-868c-028c1f572acf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:06:13.263178 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.263153    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96283224-d894-4e35-868c-028c1f572acf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "96283224-d894-4e35-868c-028c1f572acf" (UID: "96283224-d894-4e35-868c-028c1f572acf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:06:13.263449 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.263432    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-kube-api-access-bc8g7" (OuterVolumeSpecName: "kube-api-access-bc8g7") pod "96283224-d894-4e35-868c-028c1f572acf" (UID: "96283224-d894-4e35-868c-028c1f572acf"). InnerVolumeSpecName "kube-api-access-bc8g7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:06:13.269543 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.269519    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96283224-d894-4e35-868c-028c1f572acf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "96283224-d894-4e35-868c-028c1f572acf" (UID: "96283224-d894-4e35-868c-028c1f572acf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:06:13.361603 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.361579    2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-registry-tls\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:06:13.361603 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.361601    2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/96283224-d894-4e35-868c-028c1f572acf-installation-pull-secrets\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:06:13.361766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.361610    2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-bound-sa-token\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:06:13.361766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.361620    2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/96283224-d894-4e35-868c-028c1f572acf-image-registry-private-configuration\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:06:13.361766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.361628    2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/96283224-d894-4e35-868c-028c1f572acf-ca-trust-extracted\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:06:13.361766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.361638    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bc8g7\" (UniqueName: \"kubernetes.io/projected/96283224-d894-4e35-868c-028c1f572acf-kube-api-access-bc8g7\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:06:13.361766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:13.361647    2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/96283224-d894-4e35-868c-028c1f572acf-registry-certificates\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:06:14.091183 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:14.091154    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8"
Jul 28 22:06:14.091694 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:14.091192    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b9f75cc74-f72j8" event={"ID":"96283224-d894-4e35-868c-028c1f572acf","Type":"ContainerDied","Data":"45e0eb427814eeda4f7fd41e9e63a30331565c88507f848a270fc6aa156618f4"}
Jul 28 22:06:14.091694 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:14.091251    2583 scope.go:117] "RemoveContainer" containerID="fb2375659d0f936db667de7f2fb590442b4c2d643e7fa709c320f5b7fead13c7"
Jul 28 22:06:14.111753 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:14.111728    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b9f75cc74-f72j8"]
Jul 28 22:06:14.115686 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:14.115663    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7b9f75cc74-f72j8"]
Jul 28 22:06:14.431043 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:14.431011    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96283224-d894-4e35-868c-028c1f572acf" path="/var/lib/kubelet/pods/96283224-d894-4e35-868c-028c1f572acf/volumes"
Jul 28 22:06:15.603110 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:15.603076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:06:15.704099 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:15.704047    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:06:27.139910 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:27.139872    2583 generic.go:356] "Generic (PLEG): container finished" podID="1465e125-b955-4c39-bc95-67661bd1f3a3" containerID="6315c58c75f240345a80b55d8bd549a7081e1e92f31b88cda0599205360f5730" exitCode=0
Jul 28 22:06:27.140332 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:27.139946    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-77hcw" event={"ID":"1465e125-b955-4c39-bc95-67661bd1f3a3","Type":"ContainerDied","Data":"6315c58c75f240345a80b55d8bd549a7081e1e92f31b88cda0599205360f5730"}
Jul 28 22:06:27.140332 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:27.140304    2583 scope.go:117] "RemoveContainer" containerID="6315c58c75f240345a80b55d8bd549a7081e1e92f31b88cda0599205360f5730"
Jul 28 22:06:28.146789 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:28.146756    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-77hcw" event={"ID":"1465e125-b955-4c39-bc95-67661bd1f3a3","Type":"ContainerStarted","Data":"985c1dd69d187dcb441de072e6ec0da3fe1bd65d16e2dc262485bed138f988b0"}
Jul 28 22:06:37.204741 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:06:37.204702    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:09.925105 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:09.925007    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:10.328034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:10.327953    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:11.519689 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:11.519652    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:15.014568 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.014534    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-59b996856f-7qjkh"]
Jul 28 22:07:15.014972 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.014832    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96283224-d894-4e35-868c-028c1f572acf" containerName="registry"
Jul 28 22:07:15.014972 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.014844    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="96283224-d894-4e35-868c-028c1f572acf" containerName="registry"
Jul 28 22:07:15.014972 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.014906    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="96283224-d894-4e35-868c-028c1f572acf" containerName="registry"
Jul 28 22:07:15.017947 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.017930    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.020262 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.020229    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 22:07:15.020398 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.020259    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 22:07:15.020398 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.020281    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 22:07:15.020398 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.020298    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 22:07:15.020398 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.020298    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-29ngh\""
Jul 28 22:07:15.020398 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.020390    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 22:07:15.026373 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.026341    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 22:07:15.030938 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.030910    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-59b996856f-7qjkh"]
Jul 28 22:07:15.141667 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.141629    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aa44009-6562-41c7-924d-8a530fd4d38d-serving-certs-ca-bundle\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.141869 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.141684    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7aa44009-6562-41c7-924d-8a530fd4d38d-telemeter-client-tls\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.141869 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.141789    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7aa44009-6562-41c7-924d-8a530fd4d38d-secret-telemeter-client\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.141869 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.141840    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7aa44009-6562-41c7-924d-8a530fd4d38d-federate-client-tls\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.141982 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.141878    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7aa44009-6562-41c7-924d-8a530fd4d38d-metrics-client-ca\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.141982 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.141919    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7aa44009-6562-41c7-924d-8a530fd4d38d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.141982 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.141954    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aa44009-6562-41c7-924d-8a530fd4d38d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.141982 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.141976    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkbqr\" (UniqueName: \"kubernetes.io/projected/7aa44009-6562-41c7-924d-8a530fd4d38d-kube-api-access-vkbqr\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.242898 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.242864    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aa44009-6562-41c7-924d-8a530fd4d38d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.242898 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.242899    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkbqr\" (UniqueName: \"kubernetes.io/projected/7aa44009-6562-41c7-924d-8a530fd4d38d-kube-api-access-vkbqr\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.243115 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.242920    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aa44009-6562-41c7-924d-8a530fd4d38d-serving-certs-ca-bundle\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.243115 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.242947    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7aa44009-6562-41c7-924d-8a530fd4d38d-telemeter-client-tls\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.243115 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.242972    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7aa44009-6562-41c7-924d-8a530fd4d38d-secret-telemeter-client\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.243115 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.242986    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7aa44009-6562-41c7-924d-8a530fd4d38d-federate-client-tls\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.243307 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.243129    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7aa44009-6562-41c7-924d-8a530fd4d38d-metrics-client-ca\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.243307 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.243199    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7aa44009-6562-41c7-924d-8a530fd4d38d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.243829 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.243777    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aa44009-6562-41c7-924d-8a530fd4d38d-serving-certs-ca-bundle\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.243957 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.243869    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7aa44009-6562-41c7-924d-8a530fd4d38d-metrics-client-ca\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.244022 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.243877    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aa44009-6562-41c7-924d-8a530fd4d38d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.245727 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.245704    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7aa44009-6562-41c7-924d-8a530fd4d38d-telemeter-client-tls\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.245850 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.245744    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7aa44009-6562-41c7-924d-8a530fd4d38d-secret-telemeter-client\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.246010 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.245987    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7aa44009-6562-41c7-924d-8a530fd4d38d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.246064 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.246033    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7aa44009-6562-41c7-924d-8a530fd4d38d-federate-client-tls\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.250271 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.250243    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkbqr\" (UniqueName: \"kubernetes.io/projected/7aa44009-6562-41c7-924d-8a530fd4d38d-kube-api-access-vkbqr\") pod \"telemeter-client-59b996856f-7qjkh\" (UID: \"7aa44009-6562-41c7-924d-8a530fd4d38d\") " pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.328762 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.328691    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh"
Jul 28 22:07:15.460083 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:15.460051    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-59b996856f-7qjkh"]
Jul 28 22:07:15.464549 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:07:15.464521    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aa44009_6562_41c7_924d_8a530fd4d38d.slice/crio-29a7332356a955cd4fd810ddf2aabfba8c256266d526ee12b1714d54a18e5e6a WatchSource:0}: Error finding container 29a7332356a955cd4fd810ddf2aabfba8c256266d526ee12b1714d54a18e5e6a: Status 404 returned error can't find the container with id 29a7332356a955cd4fd810ddf2aabfba8c256266d526ee12b1714d54a18e5e6a
Jul 28 22:07:16.321781 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:16.321742    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh" event={"ID":"7aa44009-6562-41c7-924d-8a530fd4d38d","Type":"ContainerStarted","Data":"29a7332356a955cd4fd810ddf2aabfba8c256266d526ee12b1714d54a18e5e6a"}
Jul 28 22:07:17.326739 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:17.326707    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh" event={"ID":"7aa44009-6562-41c7-924d-8a530fd4d38d","Type":"ContainerStarted","Data":"ec9e3fc4dac1143f7a1ea034f69549f48b8d2ec6526ad320b34a7d9d239f3a91"}
Jul 28 22:07:19.337373 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:19.337338    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh" event={"ID":"7aa44009-6562-41c7-924d-8a530fd4d38d","Type":"ContainerStarted","Data":"83377eefba1f1cc3afd7db656afb39cf3abc25f1e2671403d64d52a71788507a"}
Jul 28 22:07:19.337741 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:19.337419    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh" event={"ID":"7aa44009-6562-41c7-924d-8a530fd4d38d","Type":"ContainerStarted","Data":"7f148185689c7ea01fd382f12c7d1277469c1731d9a5bdfb30d712c07fc2b6e9"}
Jul 28 22:07:19.357665 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:19.357610    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-59b996856f-7qjkh" podStartSLOduration=2.481778459 podStartE2EDuration="5.357593107s" totalImagesPullingTime="2.875814648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:07:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:07:19.355176318 +0000 UTC m=+183.363041877" watchObservedRunningTime="2026-07-28 22:07:19.357593107 +0000 UTC m=+183.365458665"
Jul 28 22:07:20.573310 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.573273    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cbdcbf9c-lvbt6"]
Jul 28 22:07:20.576782 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.576754    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.590007 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.589977    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cbdcbf9c-lvbt6"]
Jul 28 22:07:20.636424 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.636380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:20.691150 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.691111    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-config\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.691343 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.691165    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-oauth-serving-cert\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.691343 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.691229    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6ndf\" (UniqueName: \"kubernetes.io/projected/c29bdf08-8efb-4f93-9827-6339e1c42aac-kube-api-access-s6ndf\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.691343 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.691270    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-service-ca\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.691343 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.691300    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-trusted-ca-bundle\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.691343 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.691331    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-oauth-config\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.691509 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.691395    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-serving-cert\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.745648 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.745603    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:20.791989 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.791962    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-oauth-serving-cert\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.792103 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.791997    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6ndf\" (UniqueName: \"kubernetes.io/projected/c29bdf08-8efb-4f93-9827-6339e1c42aac-kube-api-access-s6ndf\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.792103 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.792018    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-service-ca\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.792206 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.792138    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-trusted-ca-bundle\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.792206 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.792189    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-oauth-config\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.792304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.792250    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-serving-cert\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.792304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.792285    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-config\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.792764 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.792739    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-service-ca\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.792914 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.792847    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-oauth-serving-cert\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.793021 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.792926    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-config\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.793021 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.792962    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-trusted-ca-bundle\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.794777 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.794750    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-oauth-config\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.794881 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.794833    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-serving-cert\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.799613 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.799596    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6ndf\" (UniqueName: \"kubernetes.io/projected/c29bdf08-8efb-4f93-9827-6339e1c42aac-kube-api-access-s6ndf\") pod \"console-cbdcbf9c-lvbt6\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") " pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:20.887356 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:20.887325    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:21.014059 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:21.013912    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cbdcbf9c-lvbt6"]
Jul 28 22:07:21.016594 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:07:21.016564    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29bdf08_8efb_4f93_9827_6339e1c42aac.slice/crio-f82795b47f31dfb85f9f10a6e3dc30469d2309d2b7de46c2f3ff9df4d2183dd9 WatchSource:0}: Error finding container f82795b47f31dfb85f9f10a6e3dc30469d2309d2b7de46c2f3ff9df4d2183dd9: Status 404 returned error can't find the container with id f82795b47f31dfb85f9f10a6e3dc30469d2309d2b7de46c2f3ff9df4d2183dd9
Jul 28 22:07:21.347825 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:21.347711    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cbdcbf9c-lvbt6" event={"ID":"c29bdf08-8efb-4f93-9827-6339e1c42aac","Type":"ContainerStarted","Data":"dfeedd50833eaa4965d2f4d8136069c79cc408b6c98dd7b4e6cf8282ff582a9e"}
Jul 28 22:07:21.347825 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:21.347760    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cbdcbf9c-lvbt6" event={"ID":"c29bdf08-8efb-4f93-9827-6339e1c42aac","Type":"ContainerStarted","Data":"f82795b47f31dfb85f9f10a6e3dc30469d2309d2b7de46c2f3ff9df4d2183dd9"}
Jul 28 22:07:21.364776 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:21.364733    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cbdcbf9c-lvbt6" podStartSLOduration=1.36471732 podStartE2EDuration="1.36471732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:07:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:07:21.363281023 +0000 UTC m=+185.371146583" watchObservedRunningTime="2026-07-28 22:07:21.36471732 +0000 UTC m=+185.372582878"
Jul 28 22:07:25.111438 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.111391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:25.154350 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.154309    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cbdcbf9c-lvbt6"]
Jul 28 22:07:25.193598 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.193562    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:25.198643 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.198620    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86565574f-rrnkj"]
Jul 28 22:07:25.201983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.201965    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.212623 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.212601    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86565574f-rrnkj"]
Jul 28 22:07:25.334190 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.334158    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-console-config\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.334190 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.334195    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drm54\" (UniqueName: \"kubernetes.io/projected/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-kube-api-access-drm54\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.334405 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.334225    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-console-oauth-config\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.334405 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.334293    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-service-ca\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.334405 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.334328    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-oauth-serving-cert\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.334405 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.334375    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-console-serving-cert\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.334535 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.334446    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-trusted-ca-bundle\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.346183 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.346157    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:25.434953 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.434919    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-console-serving-cert\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.435127 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.434964    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-trusted-ca-bundle\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.435127 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.434986    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-console-config\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.435127 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.435102    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drm54\" (UniqueName: \"kubernetes.io/projected/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-kube-api-access-drm54\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.435243 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.435160    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-console-oauth-config\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.435243 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.435214    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-service-ca\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.435338 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.435240    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-oauth-serving-cert\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.435833 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.435786    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-console-config\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.435957 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.435879    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-oauth-serving-cert\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.435957 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.435913    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-trusted-ca-bundle\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.436071 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.435972    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-service-ca\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.437550 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.437526    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-console-serving-cert\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.437859 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.437841    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-console-oauth-config\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.443163 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.443145    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drm54\" (UniqueName: \"kubernetes.io/projected/5f47692c-c93b-4c33-ba0d-f58fc617f9ce-kube-api-access-drm54\") pod \"console-86565574f-rrnkj\" (UID: \"5f47692c-c93b-4c33-ba0d-f58fc617f9ce\") " pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.511475 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.511435    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:25.644417 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.644383    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86565574f-rrnkj"]
Jul 28 22:07:25.647299 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:07:25.647266    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f47692c_c93b_4c33_ba0d_f58fc617f9ce.slice/crio-64ce2403bc5fe3e9b762b07cc59829e085436717b149136d70edcb1e46f007db WatchSource:0}: Error finding container 64ce2403bc5fe3e9b762b07cc59829e085436717b149136d70edcb1e46f007db: Status 404 returned error can't find the container with id 64ce2403bc5fe3e9b762b07cc59829e085436717b149136d70edcb1e46f007db
Jul 28 22:07:25.944296 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:25.944219    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:26.369722 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:26.369689    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86565574f-rrnkj" event={"ID":"5f47692c-c93b-4c33-ba0d-f58fc617f9ce","Type":"ContainerStarted","Data":"c0a30e38c3f93b93782625126daa37e63b702595d4bf28d753265e8b8ccb91a2"}
Jul 28 22:07:26.370105 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:26.369737    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86565574f-rrnkj" event={"ID":"5f47692c-c93b-4c33-ba0d-f58fc617f9ce","Type":"ContainerStarted","Data":"64ce2403bc5fe3e9b762b07cc59829e085436717b149136d70edcb1e46f007db"}
Jul 28 22:07:26.388868 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:26.388826    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86565574f-rrnkj" podStartSLOduration=1.3888121199999999 podStartE2EDuration="1.38881212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:07:26.386409437 +0000 UTC m=+190.394274995" watchObservedRunningTime="2026-07-28 22:07:26.38881212 +0000 UTC m=+190.396677668"
Jul 28 22:07:30.888092 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:30.888059    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:35.512080 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:35.512038    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:35.512561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:35.512106    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:35.516880 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:35.516848    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:36.417530 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:36.417500    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86565574f-rrnkj"
Jul 28 22:07:36.460635 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:36.460598    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fd7df6c4b-ctctz"]
Jul 28 22:07:36.574309 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:36.574276    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:36.631205 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:36.631175    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:36.686587 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:36.686506    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:37.169237 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:37.169183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:37.224903 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:37.224865    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:45.609290 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:45.609246    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:45.710944 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:45.710917    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:07:50.174261 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.174195    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cbdcbf9c-lvbt6" podUID="c29bdf08-8efb-4f93-9827-6339e1c42aac" containerName="console" containerID="cri-o://dfeedd50833eaa4965d2f4d8136069c79cc408b6c98dd7b4e6cf8282ff582a9e" gracePeriod=15
Jul 28 22:07:50.410595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.410574    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:50.448223 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.448157    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-trusted-ca-bundle\") pod \"c29bdf08-8efb-4f93-9827-6339e1c42aac\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") "
Jul 28 22:07:50.448223 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.448194    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-oauth-serving-cert\") pod \"c29bdf08-8efb-4f93-9827-6339e1c42aac\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") "
Jul 28 22:07:50.448409 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.448246    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-serving-cert\") pod \"c29bdf08-8efb-4f93-9827-6339e1c42aac\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") "
Jul 28 22:07:50.448409 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.448290    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-service-ca\") pod \"c29bdf08-8efb-4f93-9827-6339e1c42aac\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") "
Jul 28 22:07:50.448409 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.448312    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-oauth-config\") pod \"c29bdf08-8efb-4f93-9827-6339e1c42aac\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") "
Jul 28 22:07:50.448409 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.448345    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6ndf\" (UniqueName: \"kubernetes.io/projected/c29bdf08-8efb-4f93-9827-6339e1c42aac-kube-api-access-s6ndf\") pod \"c29bdf08-8efb-4f93-9827-6339e1c42aac\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") "
Jul 28 22:07:50.448409 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.448384    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-config\") pod \"c29bdf08-8efb-4f93-9827-6339e1c42aac\" (UID: \"c29bdf08-8efb-4f93-9827-6339e1c42aac\") "
Jul 28 22:07:50.448648 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.448612    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c29bdf08-8efb-4f93-9827-6339e1c42aac" (UID: "c29bdf08-8efb-4f93-9827-6339e1c42aac"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:07:50.448648 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.448634    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c29bdf08-8efb-4f93-9827-6339e1c42aac" (UID: "c29bdf08-8efb-4f93-9827-6339e1c42aac"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:07:50.448892 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.448760    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-service-ca" (OuterVolumeSpecName: "service-ca") pod "c29bdf08-8efb-4f93-9827-6339e1c42aac" (UID: "c29bdf08-8efb-4f93-9827-6339e1c42aac"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:07:50.449004 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.448923    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-config" (OuterVolumeSpecName: "console-config") pod "c29bdf08-8efb-4f93-9827-6339e1c42aac" (UID: "c29bdf08-8efb-4f93-9827-6339e1c42aac"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:07:50.450724 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.450685    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c29bdf08-8efb-4f93-9827-6339e1c42aac-kube-api-access-s6ndf" (OuterVolumeSpecName: "kube-api-access-s6ndf") pod "c29bdf08-8efb-4f93-9827-6339e1c42aac" (UID: "c29bdf08-8efb-4f93-9827-6339e1c42aac"). InnerVolumeSpecName "kube-api-access-s6ndf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:07:50.450939 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.450916    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c29bdf08-8efb-4f93-9827-6339e1c42aac" (UID: "c29bdf08-8efb-4f93-9827-6339e1c42aac"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:07:50.451042 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.450939    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c29bdf08-8efb-4f93-9827-6339e1c42aac" (UID: "c29bdf08-8efb-4f93-9827-6339e1c42aac"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:07:50.481034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.481007    2583 generic.go:356] "Generic (PLEG): container finished" podID="c29bdf08-8efb-4f93-9827-6339e1c42aac" containerID="dfeedd50833eaa4965d2f4d8136069c79cc408b6c98dd7b4e6cf8282ff582a9e" exitCode=0
Jul 28 22:07:50.481133 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.481064    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cbdcbf9c-lvbt6"
Jul 28 22:07:50.481133 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.481094    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cbdcbf9c-lvbt6" event={"ID":"c29bdf08-8efb-4f93-9827-6339e1c42aac","Type":"ContainerDied","Data":"dfeedd50833eaa4965d2f4d8136069c79cc408b6c98dd7b4e6cf8282ff582a9e"}
Jul 28 22:07:50.481215 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.481174    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cbdcbf9c-lvbt6" event={"ID":"c29bdf08-8efb-4f93-9827-6339e1c42aac","Type":"ContainerDied","Data":"f82795b47f31dfb85f9f10a6e3dc30469d2309d2b7de46c2f3ff9df4d2183dd9"}
Jul 28 22:07:50.481215 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.481200    2583 scope.go:117] "RemoveContainer" containerID="dfeedd50833eaa4965d2f4d8136069c79cc408b6c98dd7b4e6cf8282ff582a9e"
Jul 28 22:07:50.490389 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.490374    2583 scope.go:117] "RemoveContainer" containerID="dfeedd50833eaa4965d2f4d8136069c79cc408b6c98dd7b4e6cf8282ff582a9e"
Jul 28 22:07:50.490649 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:07:50.490629    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfeedd50833eaa4965d2f4d8136069c79cc408b6c98dd7b4e6cf8282ff582a9e\": container with ID starting with dfeedd50833eaa4965d2f4d8136069c79cc408b6c98dd7b4e6cf8282ff582a9e not found: ID does not exist" containerID="dfeedd50833eaa4965d2f4d8136069c79cc408b6c98dd7b4e6cf8282ff582a9e"
Jul 28 22:07:50.490714 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.490658    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfeedd50833eaa4965d2f4d8136069c79cc408b6c98dd7b4e6cf8282ff582a9e"} err="failed to get container status \"dfeedd50833eaa4965d2f4d8136069c79cc408b6c98dd7b4e6cf8282ff582a9e\": rpc error: code = NotFound desc = could not find container \"dfeedd50833eaa4965d2f4d8136069c79cc408b6c98dd7b4e6cf8282ff582a9e\": container with ID starting with dfeedd50833eaa4965d2f4d8136069c79cc408b6c98dd7b4e6cf8282ff582a9e not found: ID does not exist"
Jul 28 22:07:50.502474 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.502453    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cbdcbf9c-lvbt6"]
Jul 28 22:07:50.506605 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.506585    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cbdcbf9c-lvbt6"]
Jul 28 22:07:50.548997 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.548973    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-serving-cert\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:07:50.548997 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.548995    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:07:50.549126 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.549005    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-oauth-config\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:07:50.549126 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.549014    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6ndf\" (UniqueName: \"kubernetes.io/projected/c29bdf08-8efb-4f93-9827-6339e1c42aac-kube-api-access-s6ndf\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:07:50.549126 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.549023    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-console-config\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:07:50.549126 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.549030    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-trusted-ca-bundle\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:07:50.549126 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:50.549038    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c29bdf08-8efb-4f93-9827-6339e1c42aac-oauth-serving-cert\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:07:52.430716 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:07:52.430685    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c29bdf08-8efb-4f93-9827-6339e1c42aac" path="/var/lib/kubelet/pods/c29bdf08-8efb-4f93-9827-6339e1c42aac/volumes"
Jul 28 22:08:01.483809 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.483753    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6fd7df6c4b-ctctz" podUID="e957a432-274a-4a51-b300-032985eef46a" containerName="console" containerID="cri-o://b442016faa64b330be1cab5e54e8efb5e0bb1e415d921f5f6a31a97edc580e4a" gracePeriod=15
Jul 28 22:08:01.749966 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.749940    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:08:01.840998 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.840960    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmmsl\" (UniqueName: \"kubernetes.io/projected/e957a432-274a-4a51-b300-032985eef46a-kube-api-access-tmmsl\") pod \"e957a432-274a-4a51-b300-032985eef46a\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") "
Jul 28 22:08:01.841162 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.841006    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e957a432-274a-4a51-b300-032985eef46a-console-serving-cert\") pod \"e957a432-274a-4a51-b300-032985eef46a\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") "
Jul 28 22:08:01.841162 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.841069    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-service-ca\") pod \"e957a432-274a-4a51-b300-032985eef46a\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") "
Jul 28 22:08:01.841162 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.841100    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-oauth-serving-cert\") pod \"e957a432-274a-4a51-b300-032985eef46a\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") "
Jul 28 22:08:01.841162 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.841152    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e957a432-274a-4a51-b300-032985eef46a-console-oauth-config\") pod \"e957a432-274a-4a51-b300-032985eef46a\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") "
Jul 28 22:08:01.841384 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.841180    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-trusted-ca-bundle\") pod \"e957a432-274a-4a51-b300-032985eef46a\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") "
Jul 28 22:08:01.841384 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.841204    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-console-config\") pod \"e957a432-274a-4a51-b300-032985eef46a\" (UID: \"e957a432-274a-4a51-b300-032985eef46a\") "
Jul 28 22:08:01.841512 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.841483    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-service-ca" (OuterVolumeSpecName: "service-ca") pod "e957a432-274a-4a51-b300-032985eef46a" (UID: "e957a432-274a-4a51-b300-032985eef46a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:08:01.841618 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.841550    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e957a432-274a-4a51-b300-032985eef46a" (UID: "e957a432-274a-4a51-b300-032985eef46a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:08:01.841692 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.841670    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e957a432-274a-4a51-b300-032985eef46a" (UID: "e957a432-274a-4a51-b300-032985eef46a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:08:01.841749 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.841727    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-console-config" (OuterVolumeSpecName: "console-config") pod "e957a432-274a-4a51-b300-032985eef46a" (UID: "e957a432-274a-4a51-b300-032985eef46a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:08:01.843204 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.843180    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e957a432-274a-4a51-b300-032985eef46a-kube-api-access-tmmsl" (OuterVolumeSpecName: "kube-api-access-tmmsl") pod "e957a432-274a-4a51-b300-032985eef46a" (UID: "e957a432-274a-4a51-b300-032985eef46a"). InnerVolumeSpecName "kube-api-access-tmmsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:08:01.843293 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.843214    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e957a432-274a-4a51-b300-032985eef46a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e957a432-274a-4a51-b300-032985eef46a" (UID: "e957a432-274a-4a51-b300-032985eef46a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:08:01.843293 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.843236    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e957a432-274a-4a51-b300-032985eef46a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e957a432-274a-4a51-b300-032985eef46a" (UID: "e957a432-274a-4a51-b300-032985eef46a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:08:01.942012 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.941975    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e957a432-274a-4a51-b300-032985eef46a-console-oauth-config\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:08:01.942012 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.942007    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-trusted-ca-bundle\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:08:01.942012 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.942016    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-console-config\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:08:01.942236 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.942024    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmmsl\" (UniqueName: \"kubernetes.io/projected/e957a432-274a-4a51-b300-032985eef46a-kube-api-access-tmmsl\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:08:01.942236 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.942035    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e957a432-274a-4a51-b300-032985eef46a-console-serving-cert\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:08:01.942236 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.942043    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:08:01.942236 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:01.942054    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e957a432-274a-4a51-b300-032985eef46a-oauth-serving-cert\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:08:02.531227 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:02.531192    2583 generic.go:356] "Generic (PLEG): container finished" podID="e957a432-274a-4a51-b300-032985eef46a" containerID="b442016faa64b330be1cab5e54e8efb5e0bb1e415d921f5f6a31a97edc580e4a" exitCode=0
Jul 28 22:08:02.531722 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:02.531263    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fd7df6c4b-ctctz"
Jul 28 22:08:02.531722 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:02.531287    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fd7df6c4b-ctctz" event={"ID":"e957a432-274a-4a51-b300-032985eef46a","Type":"ContainerDied","Data":"b442016faa64b330be1cab5e54e8efb5e0bb1e415d921f5f6a31a97edc580e4a"}
Jul 28 22:08:02.531722 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:02.531362    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fd7df6c4b-ctctz" event={"ID":"e957a432-274a-4a51-b300-032985eef46a","Type":"ContainerDied","Data":"4c48800dc550b22b65a7beed760d125d581287044a54c60c5bd02de6dda4f3ee"}
Jul 28 22:08:02.531722 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:02.531383    2583 scope.go:117] "RemoveContainer" containerID="b442016faa64b330be1cab5e54e8efb5e0bb1e415d921f5f6a31a97edc580e4a"
Jul 28 22:08:02.539437 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:02.539413    2583 scope.go:117] "RemoveContainer" containerID="b442016faa64b330be1cab5e54e8efb5e0bb1e415d921f5f6a31a97edc580e4a"
Jul 28 22:08:02.539697 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:08:02.539674    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b442016faa64b330be1cab5e54e8efb5e0bb1e415d921f5f6a31a97edc580e4a\": container with ID starting with b442016faa64b330be1cab5e54e8efb5e0bb1e415d921f5f6a31a97edc580e4a not found: ID does not exist" containerID="b442016faa64b330be1cab5e54e8efb5e0bb1e415d921f5f6a31a97edc580e4a"
Jul 28 22:08:02.539744 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:02.539707    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b442016faa64b330be1cab5e54e8efb5e0bb1e415d921f5f6a31a97edc580e4a"} err="failed to get container status \"b442016faa64b330be1cab5e54e8efb5e0bb1e415d921f5f6a31a97edc580e4a\": rpc error: code = NotFound desc = could not find container \"b442016faa64b330be1cab5e54e8efb5e0bb1e415d921f5f6a31a97edc580e4a\": container with ID starting with b442016faa64b330be1cab5e54e8efb5e0bb1e415d921f5f6a31a97edc580e4a not found: ID does not exist"
Jul 28 22:08:02.545977 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:02.545955    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fd7df6c4b-ctctz"]
Jul 28 22:08:02.549788 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:02.549770    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6fd7df6c4b-ctctz"]
Jul 28 22:08:04.430850 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:04.430818    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e957a432-274a-4a51-b300-032985eef46a" path="/var/lib/kubelet/pods/e957a432-274a-4a51-b300-032985eef46a/volumes"
Jul 28 22:08:19.522706 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:19.522672    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:08:29.489185 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:29.489149    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:08:29.538773 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:29.538745    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:08:31.049477 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.049438    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:08:31.144304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.144259    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:08:31.699586 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.699554    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"]
Jul 28 22:08:31.699882 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.699868    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29bdf08-8efb-4f93-9827-6339e1c42aac" containerName="console"
Jul 28 22:08:31.699882 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.699882    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29bdf08-8efb-4f93-9827-6339e1c42aac" containerName="console"
Jul 28 22:08:31.699976 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.699892    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e957a432-274a-4a51-b300-032985eef46a" containerName="console"
Jul 28 22:08:31.699976 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.699900    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e957a432-274a-4a51-b300-032985eef46a" containerName="console"
Jul 28 22:08:31.699976 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.699964    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e957a432-274a-4a51-b300-032985eef46a" containerName="console"
Jul 28 22:08:31.699976 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.699974    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c29bdf08-8efb-4f93-9827-6339e1c42aac" containerName="console"
Jul 28 22:08:31.704427 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.704408    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"
Jul 28 22:08:31.706481 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.706459    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 22:08:31.706605 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.706498    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xxvmm\""
Jul 28 22:08:31.707100 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.707082    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 22:08:31.712941 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.712918    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"]
Jul 28 22:08:31.879613 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.879581    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8549\" (UniqueName: \"kubernetes.io/projected/652478d7-3b6f-49c6-a7d2-88a017e84129-kube-api-access-f8549\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr\" (UID: \"652478d7-3b6f-49c6-a7d2-88a017e84129\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"
Jul 28 22:08:31.879789 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.879633    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/652478d7-3b6f-49c6-a7d2-88a017e84129-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr\" (UID: \"652478d7-3b6f-49c6-a7d2-88a017e84129\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"
Jul 28 22:08:31.879789 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.879669    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/652478d7-3b6f-49c6-a7d2-88a017e84129-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr\" (UID: \"652478d7-3b6f-49c6-a7d2-88a017e84129\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"
Jul 28 22:08:31.980737 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.980665    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/652478d7-3b6f-49c6-a7d2-88a017e84129-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr\" (UID: \"652478d7-3b6f-49c6-a7d2-88a017e84129\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"
Jul 28 22:08:31.980737 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.980701    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/652478d7-3b6f-49c6-a7d2-88a017e84129-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr\" (UID: \"652478d7-3b6f-49c6-a7d2-88a017e84129\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"
Jul 28 22:08:31.980968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.980826    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8549\" (UniqueName: \"kubernetes.io/projected/652478d7-3b6f-49c6-a7d2-88a017e84129-kube-api-access-f8549\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr\" (UID: \"652478d7-3b6f-49c6-a7d2-88a017e84129\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"
Jul 28 22:08:31.981131 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.981112    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/652478d7-3b6f-49c6-a7d2-88a017e84129-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr\" (UID: \"652478d7-3b6f-49c6-a7d2-88a017e84129\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"
Jul 28 22:08:31.981194 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.981136    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/652478d7-3b6f-49c6-a7d2-88a017e84129-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr\" (UID: \"652478d7-3b6f-49c6-a7d2-88a017e84129\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"
Jul 28 22:08:31.988606 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:31.988582    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8549\" (UniqueName: \"kubernetes.io/projected/652478d7-3b6f-49c6-a7d2-88a017e84129-kube-api-access-f8549\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr\" (UID: \"652478d7-3b6f-49c6-a7d2-88a017e84129\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"
Jul 28 22:08:32.013540 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:32.013517    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"
Jul 28 22:08:32.138345 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:32.138319    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"]
Jul 28 22:08:32.140246 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:08:32.140218    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652478d7_3b6f_49c6_a7d2_88a017e84129.slice/crio-6ab72f08b09e73cd417cf50ff7846780cd6132733ccbc6950b19f4f20a94dd13 WatchSource:0}: Error finding container 6ab72f08b09e73cd417cf50ff7846780cd6132733ccbc6950b19f4f20a94dd13: Status 404 returned error can't find the container with id 6ab72f08b09e73cd417cf50ff7846780cd6132733ccbc6950b19f4f20a94dd13
Jul 28 22:08:32.649420 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:32.649387    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr" event={"ID":"652478d7-3b6f-49c6-a7d2-88a017e84129","Type":"ContainerStarted","Data":"6ab72f08b09e73cd417cf50ff7846780cd6132733ccbc6950b19f4f20a94dd13"}
Jul 28 22:08:38.675514 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:38.675480    2583 generic.go:356] "Generic (PLEG): container finished" podID="652478d7-3b6f-49c6-a7d2-88a017e84129" containerID="5eac0eb3c89c2e84253b908daac014a60fcdc8281151df0cf4284bf69aded228" exitCode=0
Jul 28 22:08:38.675912 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:38.675568    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr" event={"ID":"652478d7-3b6f-49c6-a7d2-88a017e84129","Type":"ContainerDied","Data":"5eac0eb3c89c2e84253b908daac014a60fcdc8281151df0cf4284bf69aded228"}
Jul 28 22:08:42.699481 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:42.699443    2583 generic.go:356] "Generic (PLEG): container finished" podID="652478d7-3b6f-49c6-a7d2-88a017e84129" containerID="5ab4869a5d7588e7715c36eb1ee70f5557901588ce3648b3ac539f72bddc8fdb" exitCode=0
Jul 28 22:08:42.699899 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:42.699513    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr" event={"ID":"652478d7-3b6f-49c6-a7d2-88a017e84129","Type":"ContainerDied","Data":"5ab4869a5d7588e7715c36eb1ee70f5557901588ce3648b3ac539f72bddc8fdb"}
Jul 28 22:08:49.732865 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:49.732827    2583 generic.go:356] "Generic (PLEG): container finished" podID="652478d7-3b6f-49c6-a7d2-88a017e84129" containerID="bf03c7f8ecac74a5baf4bdee86985961fbfcb42e4423b1b6fa2448d7644a52c6" exitCode=0
Jul 28 22:08:49.733225 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:49.732948    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr" event={"ID":"652478d7-3b6f-49c6-a7d2-88a017e84129","Type":"ContainerDied","Data":"bf03c7f8ecac74a5baf4bdee86985961fbfcb42e4423b1b6fa2448d7644a52c6"}
Jul 28 22:08:50.855832 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:50.855805    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"
Jul 28 22:08:50.924503 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:50.924478    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8549\" (UniqueName: \"kubernetes.io/projected/652478d7-3b6f-49c6-a7d2-88a017e84129-kube-api-access-f8549\") pod \"652478d7-3b6f-49c6-a7d2-88a017e84129\" (UID: \"652478d7-3b6f-49c6-a7d2-88a017e84129\") "
Jul 28 22:08:50.924635 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:50.924551    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/652478d7-3b6f-49c6-a7d2-88a017e84129-util\") pod \"652478d7-3b6f-49c6-a7d2-88a017e84129\" (UID: \"652478d7-3b6f-49c6-a7d2-88a017e84129\") "
Jul 28 22:08:50.924635 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:50.924581    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/652478d7-3b6f-49c6-a7d2-88a017e84129-bundle\") pod \"652478d7-3b6f-49c6-a7d2-88a017e84129\" (UID: \"652478d7-3b6f-49c6-a7d2-88a017e84129\") "
Jul 28 22:08:50.925313 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:50.925283    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/652478d7-3b6f-49c6-a7d2-88a017e84129-bundle" (OuterVolumeSpecName: "bundle") pod "652478d7-3b6f-49c6-a7d2-88a017e84129" (UID: "652478d7-3b6f-49c6-a7d2-88a017e84129"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:08:50.926656 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:50.926628    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/652478d7-3b6f-49c6-a7d2-88a017e84129-kube-api-access-f8549" (OuterVolumeSpecName: "kube-api-access-f8549") pod "652478d7-3b6f-49c6-a7d2-88a017e84129" (UID: "652478d7-3b6f-49c6-a7d2-88a017e84129"). InnerVolumeSpecName "kube-api-access-f8549". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:08:50.928742 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:50.928722    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/652478d7-3b6f-49c6-a7d2-88a017e84129-util" (OuterVolumeSpecName: "util") pod "652478d7-3b6f-49c6-a7d2-88a017e84129" (UID: "652478d7-3b6f-49c6-a7d2-88a017e84129"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:08:51.025784 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:51.025705    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8549\" (UniqueName: \"kubernetes.io/projected/652478d7-3b6f-49c6-a7d2-88a017e84129-kube-api-access-f8549\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:08:51.025784 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:51.025736    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/652478d7-3b6f-49c6-a7d2-88a017e84129-util\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:08:51.025784 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:51.025746    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/652478d7-3b6f-49c6-a7d2-88a017e84129-bundle\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:08:51.742138 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:51.742105    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr" event={"ID":"652478d7-3b6f-49c6-a7d2-88a017e84129","Type":"ContainerDied","Data":"6ab72f08b09e73cd417cf50ff7846780cd6132733ccbc6950b19f4f20a94dd13"}
Jul 28 22:08:51.742300 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:51.742149    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ab72f08b09e73cd417cf50ff7846780cd6132733ccbc6950b19f4f20a94dd13"
Jul 28 22:08:51.742300 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:51.742115    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540799pr"
Jul 28 22:08:57.731941 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:57.731906    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:08:58.573063 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:58.573029    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:08:59.693875 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.693842    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj"]
Jul 28 22:08:59.694232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.694122    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="652478d7-3b6f-49c6-a7d2-88a017e84129" containerName="extract"
Jul 28 22:08:59.694232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.694132    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="652478d7-3b6f-49c6-a7d2-88a017e84129" containerName="extract"
Jul 28 22:08:59.694232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.694145    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="652478d7-3b6f-49c6-a7d2-88a017e84129" containerName="util"
Jul 28 22:08:59.694232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.694151    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="652478d7-3b6f-49c6-a7d2-88a017e84129" containerName="util"
Jul 28 22:08:59.694232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.694157    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="652478d7-3b6f-49c6-a7d2-88a017e84129" containerName="pull"
Jul 28 22:08:59.694232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.694162    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="652478d7-3b6f-49c6-a7d2-88a017e84129" containerName="pull"
Jul 28 22:08:59.694232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.694215    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="652478d7-3b6f-49c6-a7d2-88a017e84129" containerName="extract"
Jul 28 22:08:59.699094 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.699075    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj"
Jul 28 22:08:59.701171 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.701138    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 28 22:08:59.701298 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.701197    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-9ch79\""
Jul 28 22:08:59.701298 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.701229    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 28 22:08:59.709129 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.709105    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj"]
Jul 28 22:08:59.784915 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.784888    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f0a5152-84a1-4cee-af91-b7eb3cf88850-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fqskj\" (UID: \"4f0a5152-84a1-4cee-af91-b7eb3cf88850\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj"
Jul 28 22:08:59.785035 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.784920    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhxx4\" (UniqueName: \"kubernetes.io/projected/4f0a5152-84a1-4cee-af91-b7eb3cf88850-kube-api-access-jhxx4\") pod \"cert-manager-operator-controller-manager-55c444bf86-fqskj\" (UID: \"4f0a5152-84a1-4cee-af91-b7eb3cf88850\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj"
Jul 28 22:08:59.885261 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.885239    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f0a5152-84a1-4cee-af91-b7eb3cf88850-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fqskj\" (UID: \"4f0a5152-84a1-4cee-af91-b7eb3cf88850\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj"
Jul 28 22:08:59.885365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.885266    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhxx4\" (UniqueName: \"kubernetes.io/projected/4f0a5152-84a1-4cee-af91-b7eb3cf88850-kube-api-access-jhxx4\") pod \"cert-manager-operator-controller-manager-55c444bf86-fqskj\" (UID: \"4f0a5152-84a1-4cee-af91-b7eb3cf88850\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj"
Jul 28 22:08:59.885596 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.885578    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f0a5152-84a1-4cee-af91-b7eb3cf88850-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fqskj\" (UID: \"4f0a5152-84a1-4cee-af91-b7eb3cf88850\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj"
Jul 28 22:08:59.892508 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:08:59.892490    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhxx4\" (UniqueName: \"kubernetes.io/projected/4f0a5152-84a1-4cee-af91-b7eb3cf88850-kube-api-access-jhxx4\") pod \"cert-manager-operator-controller-manager-55c444bf86-fqskj\" (UID: \"4f0a5152-84a1-4cee-af91-b7eb3cf88850\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj"
Jul 28 22:09:00.009818 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:00.009715    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj"
Jul 28 22:09:00.142292 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:00.142269    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj"]
Jul 28 22:09:00.145162 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:09:00.145130    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f0a5152_84a1_4cee_af91_b7eb3cf88850.slice/crio-f24bf02bb998b9969039891ede02b7917ac6eb658a323ed3e7646d8f27ecde01 WatchSource:0}: Error finding container f24bf02bb998b9969039891ede02b7917ac6eb658a323ed3e7646d8f27ecde01: Status 404 returned error can't find the container with id f24bf02bb998b9969039891ede02b7917ac6eb658a323ed3e7646d8f27ecde01
Jul 28 22:09:00.778961 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:00.778924    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj" event={"ID":"4f0a5152-84a1-4cee-af91-b7eb3cf88850","Type":"ContainerStarted","Data":"f24bf02bb998b9969039891ede02b7917ac6eb658a323ed3e7646d8f27ecde01"}
Jul 28 22:09:01.443381 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:01.443345    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:01.543006 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:01.542970    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:02.788879 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:02.788838    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj" event={"ID":"4f0a5152-84a1-4cee-af91-b7eb3cf88850","Type":"ContainerStarted","Data":"ef3c56e1d291be79c5c79921b43f215593c400ef063c3059a743af54438ab5b3"}
Jul 28 22:09:02.824018 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:02.823965    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fqskj" podStartSLOduration=1.921449955 podStartE2EDuration="3.823950133s" totalImagesPullingTime="1.902500178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:09:02.82195784 +0000 UTC m=+286.829823408" watchObservedRunningTime="2026-07-28 22:09:02.823950133 +0000 UTC m=+286.831815691"
Jul 28 22:09:03.223463 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.223429    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:03.431111 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.431079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:03.690744 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.690706    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"]
Jul 28 22:09:03.694403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.694378    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"
Jul 28 22:09:03.696306 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.696285    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 22:09:03.696397 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.696332    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xxvmm\""
Jul 28 22:09:03.696824 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.696806    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 22:09:03.712528 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.712508    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"]
Jul 28 22:09:03.713191 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.713171    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d496eace-250f-4fef-9edd-ed4c8b65b4a6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz\" (UID: \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"
Jul 28 22:09:03.713283 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.713209    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d496eace-250f-4fef-9edd-ed4c8b65b4a6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz\" (UID: \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"
Jul 28 22:09:03.713283 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.713245    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4t7j\" (UniqueName: \"kubernetes.io/projected/d496eace-250f-4fef-9edd-ed4c8b65b4a6-kube-api-access-l4t7j\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz\" (UID: \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"
Jul 28 22:09:03.764893 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.764855    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:03.813625 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.813595    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d496eace-250f-4fef-9edd-ed4c8b65b4a6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz\" (UID: \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"
Jul 28 22:09:03.813625 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.813628    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d496eace-250f-4fef-9edd-ed4c8b65b4a6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz\" (UID: \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"
Jul 28 22:09:03.814031 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.813660    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4t7j\" (UniqueName: \"kubernetes.io/projected/d496eace-250f-4fef-9edd-ed4c8b65b4a6-kube-api-access-l4t7j\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz\" (UID: \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"
Jul 28 22:09:03.814031 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.813979    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d496eace-250f-4fef-9edd-ed4c8b65b4a6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz\" (UID: \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"
Jul 28 22:09:03.814031 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.814003    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d496eace-250f-4fef-9edd-ed4c8b65b4a6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz\" (UID: \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"
Jul 28 22:09:03.826130 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.826099    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4t7j\" (UniqueName: \"kubernetes.io/projected/d496eace-250f-4fef-9edd-ed4c8b65b4a6-kube-api-access-l4t7j\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz\" (UID: \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"
Jul 28 22:09:03.940228 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:03.940195    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:04.003300 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:04.003228    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"
Jul 28 22:09:04.139634 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:04.139576    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"]
Jul 28 22:09:04.141856 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:09:04.141831    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd496eace_250f_4fef_9edd_ed4c8b65b4a6.slice/crio-d71f9f06b51a59f67cfc394bd4d4cbf8a05407293aceeb98e9d03870dcad29ca WatchSource:0}: Error finding container d71f9f06b51a59f67cfc394bd4d4cbf8a05407293aceeb98e9d03870dcad29ca: Status 404 returned error can't find the container with id d71f9f06b51a59f67cfc394bd4d4cbf8a05407293aceeb98e9d03870dcad29ca
Jul 28 22:09:04.798741 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:04.798706    2583 generic.go:356] "Generic (PLEG): container finished" podID="d496eace-250f-4fef-9edd-ed4c8b65b4a6" containerID="ed066d962c849f5c00cfff267ec0602cd31e5197874ee5fe4fb6b353a7755e63" exitCode=0
Jul 28 22:09:04.798939 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:04.798791    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz" event={"ID":"d496eace-250f-4fef-9edd-ed4c8b65b4a6","Type":"ContainerDied","Data":"ed066d962c849f5c00cfff267ec0602cd31e5197874ee5fe4fb6b353a7755e63"}
Jul 28 22:09:04.798939 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:04.798855    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz" event={"ID":"d496eace-250f-4fef-9edd-ed4c8b65b4a6","Type":"ContainerStarted","Data":"d71f9f06b51a59f67cfc394bd4d4cbf8a05407293aceeb98e9d03870dcad29ca"}
Jul 28 22:09:06.085159 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:06.085121    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:06.301694 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:06.301657    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:07.812884 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:07.812851    2583 generic.go:356] "Generic (PLEG): container finished" podID="d496eace-250f-4fef-9edd-ed4c8b65b4a6" containerID="4399914f55402725c10e48fb5c9e7c78b2e64c67faa4625da2294ea6f698a70a" exitCode=0
Jul 28 22:09:07.813289 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:07.812929    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz" event={"ID":"d496eace-250f-4fef-9edd-ed4c8b65b4a6","Type":"ContainerDied","Data":"4399914f55402725c10e48fb5c9e7c78b2e64c67faa4625da2294ea6f698a70a"}
Jul 28 22:09:08.356666 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:08.356631    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:08.454612 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:08.454573    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:08.818848 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:08.818809    2583 generic.go:356] "Generic (PLEG): container finished" podID="d496eace-250f-4fef-9edd-ed4c8b65b4a6" containerID="cdb7ad37840052213a9d3188ad76de076d079df48bc876f0506bc4881a9ed692" exitCode=0
Jul 28 22:09:08.819219 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:08.818888    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz" event={"ID":"d496eace-250f-4fef-9edd-ed4c8b65b4a6","Type":"ContainerDied","Data":"cdb7ad37840052213a9d3188ad76de076d079df48bc876f0506bc4881a9ed692"}
Jul 28 22:09:09.951850 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:09.951818    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"
Jul 28 22:09:09.959950 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:09.959924    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d496eace-250f-4fef-9edd-ed4c8b65b4a6-bundle\") pod \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\" (UID: \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\") "
Jul 28 22:09:09.960069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:09.959974    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d496eace-250f-4fef-9edd-ed4c8b65b4a6-util\") pod \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\" (UID: \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\") "
Jul 28 22:09:09.960069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:09.960016    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4t7j\" (UniqueName: \"kubernetes.io/projected/d496eace-250f-4fef-9edd-ed4c8b65b4a6-kube-api-access-l4t7j\") pod \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\" (UID: \"d496eace-250f-4fef-9edd-ed4c8b65b4a6\") "
Jul 28 22:09:09.960342 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:09.960320    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d496eace-250f-4fef-9edd-ed4c8b65b4a6-bundle" (OuterVolumeSpecName: "bundle") pod "d496eace-250f-4fef-9edd-ed4c8b65b4a6" (UID: "d496eace-250f-4fef-9edd-ed4c8b65b4a6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:09:09.961928 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:09.961906    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d496eace-250f-4fef-9edd-ed4c8b65b4a6-kube-api-access-l4t7j" (OuterVolumeSpecName: "kube-api-access-l4t7j") pod "d496eace-250f-4fef-9edd-ed4c8b65b4a6" (UID: "d496eace-250f-4fef-9edd-ed4c8b65b4a6"). InnerVolumeSpecName "kube-api-access-l4t7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:09:09.964842 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:09.964808    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d496eace-250f-4fef-9edd-ed4c8b65b4a6-util" (OuterVolumeSpecName: "util") pod "d496eace-250f-4fef-9edd-ed4c8b65b4a6" (UID: "d496eace-250f-4fef-9edd-ed4c8b65b4a6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:09:10.061427 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:10.061400    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d496eace-250f-4fef-9edd-ed4c8b65b4a6-bundle\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:09:10.061427 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:10.061422    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d496eace-250f-4fef-9edd-ed4c8b65b4a6-util\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:09:10.061427 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:10.061430    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4t7j\" (UniqueName: \"kubernetes.io/projected/d496eace-250f-4fef-9edd-ed4c8b65b4a6-kube-api-access-l4t7j\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:09:10.830021 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:10.829937    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz" event={"ID":"d496eace-250f-4fef-9edd-ed4c8b65b4a6","Type":"ContainerDied","Data":"d71f9f06b51a59f67cfc394bd4d4cbf8a05407293aceeb98e9d03870dcad29ca"}
Jul 28 22:09:10.830021 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:10.829968    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flg7vz"
Jul 28 22:09:10.830021 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:10.829983    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d71f9f06b51a59f67cfc394bd4d4cbf8a05407293aceeb98e9d03870dcad29ca"
Jul 28 22:09:13.702099 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:13.702041    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:14.387029 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:14.386995    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:14.475391 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:14.475345    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:16.320081 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.320051    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:09:16.323769 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.323745    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:09:16.332200 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.332175    2583 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 22:09:16.581029 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.580833    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9"]
Jul 28 22:09:16.605245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.581150    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d496eace-250f-4fef-9edd-ed4c8b65b4a6" containerName="extract"
Jul 28 22:09:16.605245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.581160    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d496eace-250f-4fef-9edd-ed4c8b65b4a6" containerName="extract"
Jul 28 22:09:16.605245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.581170    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d496eace-250f-4fef-9edd-ed4c8b65b4a6" containerName="util"
Jul 28 22:09:16.605245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.581176    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d496eace-250f-4fef-9edd-ed4c8b65b4a6" containerName="util"
Jul 28 22:09:16.605245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.581185    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d496eace-250f-4fef-9edd-ed4c8b65b4a6" containerName="pull"
Jul 28 22:09:16.605245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.581189    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d496eace-250f-4fef-9edd-ed4c8b65b4a6" containerName="pull"
Jul 28 22:09:16.605245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.581239    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d496eace-250f-4fef-9edd-ed4c8b65b4a6" containerName="extract"
Jul 28 22:09:16.605245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.584774    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9"
Jul 28 22:09:16.605245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.586855    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-tq6h6\""
Jul 28 22:09:16.605245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.586878    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 22:09:16.605245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.587274    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 22:09:16.605245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.591328    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9"]
Jul 28 22:09:16.609656 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.609635    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lpdz\" (UniqueName: \"kubernetes.io/projected/36c779c4-7c38-4290-a263-996940b9b881-kube-api-access-2lpdz\") pod \"openshift-lws-operator-fd8ccff4c-d7ll9\" (UID: \"36c779c4-7c38-4290-a263-996940b9b881\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9"
Jul 28 22:09:16.609746 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.609673    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36c779c4-7c38-4290-a263-996940b9b881-tmp\") pod \"openshift-lws-operator-fd8ccff4c-d7ll9\" (UID: \"36c779c4-7c38-4290-a263-996940b9b881\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9"
Jul 28 22:09:16.710819 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.710760    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lpdz\" (UniqueName: \"kubernetes.io/projected/36c779c4-7c38-4290-a263-996940b9b881-kube-api-access-2lpdz\") pod \"openshift-lws-operator-fd8ccff4c-d7ll9\" (UID: \"36c779c4-7c38-4290-a263-996940b9b881\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9"
Jul 28 22:09:16.711027 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.710846    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36c779c4-7c38-4290-a263-996940b9b881-tmp\") pod \"openshift-lws-operator-fd8ccff4c-d7ll9\" (UID: \"36c779c4-7c38-4290-a263-996940b9b881\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9"
Jul 28 22:09:16.711230 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.711209    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36c779c4-7c38-4290-a263-996940b9b881-tmp\") pod \"openshift-lws-operator-fd8ccff4c-d7ll9\" (UID: \"36c779c4-7c38-4290-a263-996940b9b881\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9"
Jul 28 22:09:16.718898 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.718873    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lpdz\" (UniqueName: \"kubernetes.io/projected/36c779c4-7c38-4290-a263-996940b9b881-kube-api-access-2lpdz\") pod \"openshift-lws-operator-fd8ccff4c-d7ll9\" (UID: \"36c779c4-7c38-4290-a263-996940b9b881\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9"
Jul 28 22:09:16.903776 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:16.903741    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9"
Jul 28 22:09:17.051136 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:17.051102    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9"]
Jul 28 22:09:17.054387 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:09:17.054356    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36c779c4_7c38_4290_a263_996940b9b881.slice/crio-4d38d8737bf17edb5d4a256823686eaf31a94cc155ce2a94cc72edee7b544550 WatchSource:0}: Error finding container 4d38d8737bf17edb5d4a256823686eaf31a94cc155ce2a94cc72edee7b544550: Status 404 returned error can't find the container with id 4d38d8737bf17edb5d4a256823686eaf31a94cc155ce2a94cc72edee7b544550
Jul 28 22:09:17.860628 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:17.860585    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9" event={"ID":"36c779c4-7c38-4290-a263-996940b9b881","Type":"ContainerStarted","Data":"4d38d8737bf17edb5d4a256823686eaf31a94cc155ce2a94cc72edee7b544550"}
Jul 28 22:09:18.868003 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:18.867964    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9" event={"ID":"36c779c4-7c38-4290-a263-996940b9b881","Type":"ContainerStarted","Data":"5f47568ce0a063335e9e84790af1c9050e280d901d4a1150b3de45ed116580a4"}
Jul 28 22:09:18.883523 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:18.883473    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d7ll9" podStartSLOduration=1.345019558 podStartE2EDuration="2.88345712s" totalImagesPullingTime="1.538437562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:09:18.881705967 +0000 UTC m=+302.889571525" watchObservedRunningTime="2026-07-28 22:09:18.88345712 +0000 UTC m=+302.891322678"
Jul 28 22:09:19.679682 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:19.679651    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:21.029345 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.029311    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:21.644663 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.644632    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"]
Jul 28 22:09:21.648673 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.648657    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"
Jul 28 22:09:21.650618 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.650592    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xxvmm\""
Jul 28 22:09:21.650727 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.650593    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 22:09:21.650727 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.650592    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 22:09:21.656150 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.656128    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"]
Jul 28 22:09:21.756855 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.756823    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1da1b084-a79f-4c00-8d1e-b823083c1e9f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl\" (UID: \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"
Jul 28 22:09:21.757032 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.756876    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1da1b084-a79f-4c00-8d1e-b823083c1e9f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl\" (UID: \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"
Jul 28 22:09:21.757032 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.756905    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zj4d\" (UniqueName: \"kubernetes.io/projected/1da1b084-a79f-4c00-8d1e-b823083c1e9f-kube-api-access-4zj4d\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl\" (UID: \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"
Jul 28 22:09:21.765432 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.765401    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:21.857565 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.857526    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1da1b084-a79f-4c00-8d1e-b823083c1e9f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl\" (UID: \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"
Jul 28 22:09:21.857565 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.857572    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1da1b084-a79f-4c00-8d1e-b823083c1e9f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl\" (UID: \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"
Jul 28 22:09:21.857836 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.857594    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zj4d\" (UniqueName: \"kubernetes.io/projected/1da1b084-a79f-4c00-8d1e-b823083c1e9f-kube-api-access-4zj4d\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl\" (UID: \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"
Jul 28 22:09:21.857942 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.857921    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1da1b084-a79f-4c00-8d1e-b823083c1e9f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl\" (UID: \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"
Jul 28 22:09:21.858022 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.858001    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1da1b084-a79f-4c00-8d1e-b823083c1e9f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl\" (UID: \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"
Jul 28 22:09:21.865450 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.865425    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zj4d\" (UniqueName: \"kubernetes.io/projected/1da1b084-a79f-4c00-8d1e-b823083c1e9f-kube-api-access-4zj4d\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl\" (UID: \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"
Jul 28 22:09:21.958590 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:21.958504    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"
Jul 28 22:09:22.107471 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:22.107439    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"]
Jul 28 22:09:22.110489 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:09:22.110464    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da1b084_a79f_4c00_8d1e_b823083c1e9f.slice/crio-f41fc6a495a9699ca3aff444db28bc47532ccc9e8be417adc865b5a8cb98c23a WatchSource:0}: Error finding container f41fc6a495a9699ca3aff444db28bc47532ccc9e8be417adc865b5a8cb98c23a: Status 404 returned error can't find the container with id f41fc6a495a9699ca3aff444db28bc47532ccc9e8be417adc865b5a8cb98c23a
Jul 28 22:09:22.886946 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:22.886915    2583 generic.go:356] "Generic (PLEG): container finished" podID="1da1b084-a79f-4c00-8d1e-b823083c1e9f" containerID="768357aa396b4a7684b878665be8ff30b7e6ba1bd0e5ce14050a5cb9e0c2c9ea" exitCode=0
Jul 28 22:09:22.886946 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:22.886950    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl" event={"ID":"1da1b084-a79f-4c00-8d1e-b823083c1e9f","Type":"ContainerDied","Data":"768357aa396b4a7684b878665be8ff30b7e6ba1bd0e5ce14050a5cb9e0c2c9ea"}
Jul 28 22:09:22.887134 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:22.886995    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl" event={"ID":"1da1b084-a79f-4c00-8d1e-b823083c1e9f","Type":"ContainerStarted","Data":"f41fc6a495a9699ca3aff444db28bc47532ccc9e8be417adc865b5a8cb98c23a"}
Jul 28 22:09:22.887446 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:22.887431    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:09:24.897210 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:24.897177    2583 generic.go:356] "Generic (PLEG): container finished" podID="1da1b084-a79f-4c00-8d1e-b823083c1e9f" containerID="d2a6e9557f0af5720d68398f679215db8a8a0b21bdc56629be59e54ee6abd9e6" exitCode=0
Jul 28 22:09:24.897681 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:24.897240    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl" event={"ID":"1da1b084-a79f-4c00-8d1e-b823083c1e9f","Type":"ContainerDied","Data":"d2a6e9557f0af5720d68398f679215db8a8a0b21bdc56629be59e54ee6abd9e6"}
Jul 28 22:09:25.904348 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:25.904314    2583 generic.go:356] "Generic (PLEG): container finished" podID="1da1b084-a79f-4c00-8d1e-b823083c1e9f" containerID="735a8ebfa7a9e773ef6b0349ee5b86f187ae336295781cc95280d07fd2a5fdf0" exitCode=0
Jul 28 22:09:25.904743 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:25.904394    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl" event={"ID":"1da1b084-a79f-4c00-8d1e-b823083c1e9f","Type":"ContainerDied","Data":"735a8ebfa7a9e773ef6b0349ee5b86f187ae336295781cc95280d07fd2a5fdf0"}
Jul 28 22:09:27.031231 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:27.031210    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"
Jul 28 22:09:27.101407 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:27.101380    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1da1b084-a79f-4c00-8d1e-b823083c1e9f-bundle\") pod \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\" (UID: \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\") "
Jul 28 22:09:27.101523 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:27.101426    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zj4d\" (UniqueName: \"kubernetes.io/projected/1da1b084-a79f-4c00-8d1e-b823083c1e9f-kube-api-access-4zj4d\") pod \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\" (UID: \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\") "
Jul 28 22:09:27.101523 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:27.101455    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1da1b084-a79f-4c00-8d1e-b823083c1e9f-util\") pod \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\" (UID: \"1da1b084-a79f-4c00-8d1e-b823083c1e9f\") "
Jul 28 22:09:27.102118 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:27.102074    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da1b084-a79f-4c00-8d1e-b823083c1e9f-bundle" (OuterVolumeSpecName: "bundle") pod "1da1b084-a79f-4c00-8d1e-b823083c1e9f" (UID: "1da1b084-a79f-4c00-8d1e-b823083c1e9f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:09:27.103890 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:27.103860    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da1b084-a79f-4c00-8d1e-b823083c1e9f-kube-api-access-4zj4d" (OuterVolumeSpecName: "kube-api-access-4zj4d") pod "1da1b084-a79f-4c00-8d1e-b823083c1e9f" (UID: "1da1b084-a79f-4c00-8d1e-b823083c1e9f"). InnerVolumeSpecName "kube-api-access-4zj4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:09:27.107666 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:27.107639    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da1b084-a79f-4c00-8d1e-b823083c1e9f-util" (OuterVolumeSpecName: "util") pod "1da1b084-a79f-4c00-8d1e-b823083c1e9f" (UID: "1da1b084-a79f-4c00-8d1e-b823083c1e9f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:09:27.202014 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:27.201952    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1da1b084-a79f-4c00-8d1e-b823083c1e9f-util\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:09:27.202014 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:27.201977    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1da1b084-a79f-4c00-8d1e-b823083c1e9f-bundle\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:09:27.202014 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:27.201986    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zj4d\" (UniqueName: \"kubernetes.io/projected/1da1b084-a79f-4c00-8d1e-b823083c1e9f-kube-api-access-4zj4d\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:09:27.914444 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:27.914411    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl"
Jul 28 22:09:27.914586 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:27.914439    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftp2xl" event={"ID":"1da1b084-a79f-4c00-8d1e-b823083c1e9f","Type":"ContainerDied","Data":"f41fc6a495a9699ca3aff444db28bc47532ccc9e8be417adc865b5a8cb98c23a"}
Jul 28 22:09:27.914586 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:27.914485    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f41fc6a495a9699ca3aff444db28bc47532ccc9e8be417adc865b5a8cb98c23a"
Jul 28 22:09:34.564848 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:34.564805    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:34.674981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:34.674950    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:35.195989 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.195952    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:35.569548 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.569476    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"]
Jul 28 22:09:35.569907 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.569784    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da1b084-a79f-4c00-8d1e-b823083c1e9f" containerName="extract"
Jul 28 22:09:35.569907 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.569822    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da1b084-a79f-4c00-8d1e-b823083c1e9f" containerName="extract"
Jul 28 22:09:35.569907 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.569831    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da1b084-a79f-4c00-8d1e-b823083c1e9f" containerName="util"
Jul 28 22:09:35.569907 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.569837    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da1b084-a79f-4c00-8d1e-b823083c1e9f" containerName="util"
Jul 28 22:09:35.569907 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.569854    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da1b084-a79f-4c00-8d1e-b823083c1e9f" containerName="pull"
Jul 28 22:09:35.569907 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.569860    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da1b084-a79f-4c00-8d1e-b823083c1e9f" containerName="pull"
Jul 28 22:09:35.572871 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.570882    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da1b084-a79f-4c00-8d1e-b823083c1e9f" containerName="extract"
Jul 28 22:09:35.577316 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.577291    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:35.578058 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.578038    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.580306 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.580286    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gbw8q\""
Jul 28 22:09:35.580469 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.580450    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 22:09:35.580860 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.580843    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 22:09:35.580932 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.580870    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 22:09:35.581107 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.581094    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 22:09:35.593882 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.593857    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"]
Jul 28 22:09:35.659421 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.659391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:35.679388 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.679357    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-webhook-cert\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.679527 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.679403    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.679527 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.679497    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-manifests\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.679614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.679532    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8twt5\" (UniqueName: \"kubernetes.io/projected/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-kube-api-access-8twt5\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.679614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.679553    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-tmp\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.759461 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.759420    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:35.780481 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.780457    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-webhook-cert\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.780575 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.780493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.780686 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.780669    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-manifests\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.780740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.780715    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8twt5\" (UniqueName: \"kubernetes.io/projected/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-kube-api-access-8twt5\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.780782 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.780742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-tmp\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.781116 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.781092    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-manifests\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.783137 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.783114    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-webhook-cert\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.783232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.783176    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-tmp\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.783409 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.783391    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.790981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.790962    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8twt5\" (UniqueName: \"kubernetes.io/projected/cf38cbba-f1b2-4ca0-82ef-c5989ee07b27-kube-api-access-8twt5\") pod \"opendatahub-operator-controller-manager-d45996775-ngz2z\" (UID: \"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27\") " pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.860141 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.860067    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:35.888106 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.888072    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:35.960070 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:35.960041    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:36.032548 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:36.032527    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"]
Jul 28 22:09:36.034928 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:09:36.034901    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf38cbba_f1b2_4ca0_82ef_c5989ee07b27.slice/crio-0b949a9cc7c78696788f092d50d57e5d1fb13030b1e8d780a0204107f0eeca43 WatchSource:0}: Error finding container 0b949a9cc7c78696788f092d50d57e5d1fb13030b1e8d780a0204107f0eeca43: Status 404 returned error can't find the container with id 0b949a9cc7c78696788f092d50d57e5d1fb13030b1e8d780a0204107f0eeca43
Jul 28 22:09:36.039870 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:36.039846    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:36.112574 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:36.112485    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:36.227169 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:36.227136    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:36.317907 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:36.317859    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:36.422454 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:36.422413    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:36.959572 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:36.959527    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z" event={"ID":"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27","Type":"ContainerStarted","Data":"0b949a9cc7c78696788f092d50d57e5d1fb13030b1e8d780a0204107f0eeca43"}
Jul 28 22:09:39.974319 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:39.974281    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z" event={"ID":"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27","Type":"ContainerStarted","Data":"ce97ef5dc126b5b33284cb67b5404b3c6231589969c1e61af603faaed991f699"}
Jul 28 22:09:40.980593 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:40.980560    2583 generic.go:356] "Generic (PLEG): container finished" podID="cf38cbba-f1b2-4ca0-82ef-c5989ee07b27" containerID="ce97ef5dc126b5b33284cb67b5404b3c6231589969c1e61af603faaed991f699" exitCode=0
Jul 28 22:09:40.981018 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:40.980626    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z" event={"ID":"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27","Type":"ContainerDied","Data":"ce97ef5dc126b5b33284cb67b5404b3c6231589969c1e61af603faaed991f699"}
Jul 28 22:09:41.544809 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:41.544759    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:41.620228 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:41.620193    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:41.986787 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:41.986752    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z" event={"ID":"cf38cbba-f1b2-4ca0-82ef-c5989ee07b27","Type":"ContainerStarted","Data":"ff32521f0e4b05d186a28f6f1869626ea516dd17d6b4c971447285f67bf97fc9"}
Jul 28 22:09:41.987193 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:41.986844    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:42.009671 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:42.009618    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z" podStartSLOduration=2.479181846 podStartE2EDuration="7.00960204s" totalImagesPullingTime="3.48975526s" totalInitContainerRuntime="1.040664934s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:09:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:09:42.007684267 +0000 UTC m=+326.015549821" watchObservedRunningTime="2026-07-28 22:09:42.00960204 +0000 UTC m=+326.017467599"
Jul 28 22:09:46.743278 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:46.743239    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:46.841535 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:46.841499    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:50.102983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:50.102942    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:50.205029 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:50.204998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:52.993644 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:52.993612    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d45996775-ngz2z"
Jul 28 22:09:54.463972 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.463943    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"]
Jul 28 22:09:54.471939 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.471919    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.477651 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.477632    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 22:09:54.477760 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.477648    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4wdp4\""
Jul 28 22:09:54.477760 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.477732    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 22:09:54.477760 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.477755    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 22:09:54.488689 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.488665    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"]
Jul 28 22:09:54.547753 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.547715    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlgrv\" (UniqueName: \"kubernetes.io/projected/25e6fd4a-cec6-4160-8556-69f4ba320818-kube-api-access-rlgrv\") pod \"lws-controller-manager-64ddd775c9-chjj4\" (UID: \"25e6fd4a-cec6-4160-8556-69f4ba320818\") " pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.547952 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.547821    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/25e6fd4a-cec6-4160-8556-69f4ba320818-metrics-cert\") pod \"lws-controller-manager-64ddd775c9-chjj4\" (UID: \"25e6fd4a-cec6-4160-8556-69f4ba320818\") " pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.547952 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.547928    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25e6fd4a-cec6-4160-8556-69f4ba320818-cert\") pod \"lws-controller-manager-64ddd775c9-chjj4\" (UID: \"25e6fd4a-cec6-4160-8556-69f4ba320818\") " pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.548126 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.548003    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/25e6fd4a-cec6-4160-8556-69f4ba320818-manager-config\") pod \"lws-controller-manager-64ddd775c9-chjj4\" (UID: \"25e6fd4a-cec6-4160-8556-69f4ba320818\") " pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.606454 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.606417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:54.648512 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.648479    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/25e6fd4a-cec6-4160-8556-69f4ba320818-metrics-cert\") pod \"lws-controller-manager-64ddd775c9-chjj4\" (UID: \"25e6fd4a-cec6-4160-8556-69f4ba320818\") " pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.648512 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.648511    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25e6fd4a-cec6-4160-8556-69f4ba320818-cert\") pod \"lws-controller-manager-64ddd775c9-chjj4\" (UID: \"25e6fd4a-cec6-4160-8556-69f4ba320818\") " pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.648703 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.648549    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/25e6fd4a-cec6-4160-8556-69f4ba320818-manager-config\") pod \"lws-controller-manager-64ddd775c9-chjj4\" (UID: \"25e6fd4a-cec6-4160-8556-69f4ba320818\") " pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.648703 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.648585    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlgrv\" (UniqueName: \"kubernetes.io/projected/25e6fd4a-cec6-4160-8556-69f4ba320818-kube-api-access-rlgrv\") pod \"lws-controller-manager-64ddd775c9-chjj4\" (UID: \"25e6fd4a-cec6-4160-8556-69f4ba320818\") " pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.649339 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.649312    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/25e6fd4a-cec6-4160-8556-69f4ba320818-manager-config\") pod \"lws-controller-manager-64ddd775c9-chjj4\" (UID: \"25e6fd4a-cec6-4160-8556-69f4ba320818\") " pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.650906 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.650883    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/25e6fd4a-cec6-4160-8556-69f4ba320818-metrics-cert\") pod \"lws-controller-manager-64ddd775c9-chjj4\" (UID: \"25e6fd4a-cec6-4160-8556-69f4ba320818\") " pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.650990 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.650927    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25e6fd4a-cec6-4160-8556-69f4ba320818-cert\") pod \"lws-controller-manager-64ddd775c9-chjj4\" (UID: \"25e6fd4a-cec6-4160-8556-69f4ba320818\") " pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.669932 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.669908    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlgrv\" (UniqueName: \"kubernetes.io/projected/25e6fd4a-cec6-4160-8556-69f4ba320818-kube-api-access-rlgrv\") pod \"lws-controller-manager-64ddd775c9-chjj4\" (UID: \"25e6fd4a-cec6-4160-8556-69f4ba320818\") " pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.741777 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.741673    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:54.786760 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.786739    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:54.937682 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:09:54.937647    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25e6fd4a_cec6_4160_8556_69f4ba320818.slice/crio-698a8fe575909586a91b9ae723d722b1e9bcc1b53037c456b78777ef35d28e85 WatchSource:0}: Error finding container 698a8fe575909586a91b9ae723d722b1e9bcc1b53037c456b78777ef35d28e85: Status 404 returned error can't find the container with id 698a8fe575909586a91b9ae723d722b1e9bcc1b53037c456b78777ef35d28e85
Jul 28 22:09:54.949988 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:54.949966    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"]
Jul 28 22:09:55.050272 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:55.050195    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4" event={"ID":"25e6fd4a-cec6-4160-8556-69f4ba320818","Type":"ContainerStarted","Data":"698a8fe575909586a91b9ae723d722b1e9bcc1b53037c456b78777ef35d28e85"}
Jul 28 22:09:55.442530 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:55.442492    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:55.526633 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:55.526597    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:55.626992 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:55.626919    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:55.827339 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:55.827224    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:55.938455 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:55.938381    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:56.095843 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.095746    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"]
Jul 28 22:09:56.099899 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.099875    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"
Jul 28 22:09:56.102626 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.102288    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 22:09:56.102626 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.102392    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 22:09:56.102626 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.102549    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bvl6c\""
Jul 28 22:09:56.113357 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.113333    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"]
Jul 28 22:09:56.152651 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.152602    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:56.159778 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.159730    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41bfaa5e-c4c1-4d52-a308-f759c8afdbcf-tls-certs\") pod \"kube-auth-proxy-5cd5d84f76-xlct4\" (UID: \"41bfaa5e-c4c1-4d52-a308-f759c8afdbcf\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"
Jul 28 22:09:56.159934 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.159865    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5m9f\" (UniqueName: \"kubernetes.io/projected/41bfaa5e-c4c1-4d52-a308-f759c8afdbcf-kube-api-access-s5m9f\") pod \"kube-auth-proxy-5cd5d84f76-xlct4\" (UID: \"41bfaa5e-c4c1-4d52-a308-f759c8afdbcf\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"
Jul 28 22:09:56.159934 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.159913    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41bfaa5e-c4c1-4d52-a308-f759c8afdbcf-tmp\") pod \"kube-auth-proxy-5cd5d84f76-xlct4\" (UID: \"41bfaa5e-c4c1-4d52-a308-f759c8afdbcf\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"
Jul 28 22:09:56.261145 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.261111    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41bfaa5e-c4c1-4d52-a308-f759c8afdbcf-tls-certs\") pod \"kube-auth-proxy-5cd5d84f76-xlct4\" (UID: \"41bfaa5e-c4c1-4d52-a308-f759c8afdbcf\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"
Jul 28 22:09:56.261321 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.261201    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5m9f\" (UniqueName: \"kubernetes.io/projected/41bfaa5e-c4c1-4d52-a308-f759c8afdbcf-kube-api-access-s5m9f\") pod \"kube-auth-proxy-5cd5d84f76-xlct4\" (UID: \"41bfaa5e-c4c1-4d52-a308-f759c8afdbcf\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"
Jul 28 22:09:56.261321 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.261241    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41bfaa5e-c4c1-4d52-a308-f759c8afdbcf-tmp\") pod \"kube-auth-proxy-5cd5d84f76-xlct4\" (UID: \"41bfaa5e-c4c1-4d52-a308-f759c8afdbcf\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"
Jul 28 22:09:56.261321 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:09:56.261253    2583 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 28 22:09:56.261321 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:09:56.261319    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41bfaa5e-c4c1-4d52-a308-f759c8afdbcf-tls-certs podName:41bfaa5e-c4c1-4d52-a308-f759c8afdbcf nodeName:}" failed. No retries permitted until 2026-07-28 22:09:56.761303122 +0000 UTC m=+340.769168662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/41bfaa5e-c4c1-4d52-a308-f759c8afdbcf-tls-certs") pod "kube-auth-proxy-5cd5d84f76-xlct4" (UID: "41bfaa5e-c4c1-4d52-a308-f759c8afdbcf") : secret "kube-auth-proxy-tls" not found
Jul 28 22:09:56.263986 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.263962    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41bfaa5e-c4c1-4d52-a308-f759c8afdbcf-tmp\") pod \"kube-auth-proxy-5cd5d84f76-xlct4\" (UID: \"41bfaa5e-c4c1-4d52-a308-f759c8afdbcf\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"
Jul 28 22:09:56.272058 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.272017    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5m9f\" (UniqueName: \"kubernetes.io/projected/41bfaa5e-c4c1-4d52-a308-f759c8afdbcf-kube-api-access-s5m9f\") pod \"kube-auth-proxy-5cd5d84f76-xlct4\" (UID: \"41bfaa5e-c4c1-4d52-a308-f759c8afdbcf\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"
Jul 28 22:09:56.666893 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.666855    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:56.765300 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.765207    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41bfaa5e-c4c1-4d52-a308-f759c8afdbcf-tls-certs\") pod \"kube-auth-proxy-5cd5d84f76-xlct4\" (UID: \"41bfaa5e-c4c1-4d52-a308-f759c8afdbcf\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"
Jul 28 22:09:56.767561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.767542    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41bfaa5e-c4c1-4d52-a308-f759c8afdbcf-tls-certs\") pod \"kube-auth-proxy-5cd5d84f76-xlct4\" (UID: \"41bfaa5e-c4c1-4d52-a308-f759c8afdbcf\") " pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"
Jul 28 22:09:56.881496 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:56.881467    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:57.016241 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:57.016156    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"
Jul 28 22:09:57.062700 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:57.062669    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4" event={"ID":"25e6fd4a-cec6-4160-8556-69f4ba320818","Type":"ContainerStarted","Data":"46f08764fdc902f03b1a4a19279449db3dd3330f8e3064bf6cd5dd73f2a6b6b6"}
Jul 28 22:09:57.062872 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:57.062816    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:09:57.081567 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:57.081488    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4" podStartSLOduration=1.550657666 podStartE2EDuration="3.081468534s" totalImagesPullingTime="1.530810868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:09:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:09:57.081114701 +0000 UTC m=+341.088980280" watchObservedRunningTime="2026-07-28 22:09:57.081468534 +0000 UTC m=+341.089334105"
Jul 28 22:09:57.144991 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:57.144969    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4"]
Jul 28 22:09:57.146663 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:09:57.146638    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41bfaa5e_c4c1_4d52_a308_f759c8afdbcf.slice/crio-1d94c2b24abbe1e1fb380f9a5957bfce478c9abc2613b4e34d65f68d0de01207 WatchSource:0}: Error finding container 1d94c2b24abbe1e1fb380f9a5957bfce478c9abc2613b4e34d65f68d0de01207: Status 404 returned error can't find the container with id 1d94c2b24abbe1e1fb380f9a5957bfce478c9abc2613b4e34d65f68d0de01207
Jul 28 22:09:57.290149 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:57.290076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:57.389384 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:57.389355    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:09:58.069246 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:58.069210    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4" event={"ID":"41bfaa5e-c4c1-4d52-a308-f759c8afdbcf","Type":"ContainerStarted","Data":"1d94c2b24abbe1e1fb380f9a5957bfce478c9abc2613b4e34d65f68d0de01207"}
Jul 28 22:09:58.480358 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:09:58.480319    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:00.079965 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:00.079878    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4" event={"ID":"41bfaa5e-c4c1-4d52-a308-f759c8afdbcf","Type":"ContainerStarted","Data":"26695490eec4e1685fb1ac6edf3b368a0ff1927a24cbe5493c8df15a64d9fc4d"}
Jul 28 22:10:00.095971 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:00.095925    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5cd5d84f76-xlct4" podStartSLOduration=1.492709714 podStartE2EDuration="4.095909798s" totalImagesPullingTime="2.603200084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:09:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:00.095214482 +0000 UTC m=+344.103080062" watchObservedRunningTime="2026-07-28 22:10:00.095909798 +0000 UTC m=+344.103775356"
Jul 28 22:10:03.076811 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:03.076761    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:03.179100 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:03.179061    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:03.354554 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:03.354468    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:03.513808 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:03.513774    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:03.649247 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:03.649211    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:05.695647 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:05.695606    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:05.817175 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:05.817132    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:05.893293 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:05.893254    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:06.311430 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:06.311393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:07.014846 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:07.014810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:07.115330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:07.115284    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:07.223837 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:07.223780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:08.071430 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:08.071404    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-64ddd775c9-chjj4"
Jul 28 22:10:52.510844 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:52.510805    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:52.568275 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:52.568243    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:56.328008 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:56.327970    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:56.427632 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:56.427600    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:56.505571 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:56.505538    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:57.267975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:57.267944    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:10:57.467736 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:10:57.467703    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:03.149497 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:03.149460    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:03.209159 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:03.209120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:12.294063 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.294021    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"]
Jul 28 22:11:12.297542 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.297527    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"
Jul 28 22:11:12.299929 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.299895    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 22:11:12.299929 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.299918    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 22:11:12.300113 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.299978    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h7fc5\""
Jul 28 22:11:12.305139 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.305118    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"]
Jul 28 22:11:12.416079 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.416046    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41654737-66b5-44ea-8bbb-78c923e5af0e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn\" (UID: \"41654737-66b5-44ea-8bbb-78c923e5af0e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"
Jul 28 22:11:12.416249 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.416169    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h66pv\" (UniqueName: \"kubernetes.io/projected/41654737-66b5-44ea-8bbb-78c923e5af0e-kube-api-access-h66pv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn\" (UID: \"41654737-66b5-44ea-8bbb-78c923e5af0e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"
Jul 28 22:11:12.416249 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.416191    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41654737-66b5-44ea-8bbb-78c923e5af0e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn\" (UID: \"41654737-66b5-44ea-8bbb-78c923e5af0e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"
Jul 28 22:11:12.516987 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.516952    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h66pv\" (UniqueName: \"kubernetes.io/projected/41654737-66b5-44ea-8bbb-78c923e5af0e-kube-api-access-h66pv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn\" (UID: \"41654737-66b5-44ea-8bbb-78c923e5af0e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"
Jul 28 22:11:12.516987 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.516990    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41654737-66b5-44ea-8bbb-78c923e5af0e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn\" (UID: \"41654737-66b5-44ea-8bbb-78c923e5af0e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"
Jul 28 22:11:12.517226 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.517196    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41654737-66b5-44ea-8bbb-78c923e5af0e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn\" (UID: \"41654737-66b5-44ea-8bbb-78c923e5af0e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"
Jul 28 22:11:12.517370 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.517351    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41654737-66b5-44ea-8bbb-78c923e5af0e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn\" (UID: \"41654737-66b5-44ea-8bbb-78c923e5af0e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"
Jul 28 22:11:12.517521 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.517504    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41654737-66b5-44ea-8bbb-78c923e5af0e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn\" (UID: \"41654737-66b5-44ea-8bbb-78c923e5af0e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"
Jul 28 22:11:12.525800 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.525780    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h66pv\" (UniqueName: \"kubernetes.io/projected/41654737-66b5-44ea-8bbb-78c923e5af0e-kube-api-access-h66pv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn\" (UID: \"41654737-66b5-44ea-8bbb-78c923e5af0e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"
Jul 28 22:11:12.607754 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.607665    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"
Jul 28 22:11:12.679810 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.679764    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"]
Jul 28 22:11:12.683539 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.683516    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"
Jul 28 22:11:12.696177 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.696151    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"]
Jul 28 22:11:12.746453 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.746389    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"]
Jul 28 22:11:12.748621 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:11:12.748593    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41654737_66b5_44ea_8bbb_78c923e5af0e.slice/crio-9539f2311803a8e34e960ad8daceab297c2bd4e3b502ade849f5d8baaa55f2d9 WatchSource:0}: Error finding container 9539f2311803a8e34e960ad8daceab297c2bd4e3b502ade849f5d8baaa55f2d9: Status 404 returned error can't find the container with id 9539f2311803a8e34e960ad8daceab297c2bd4e3b502ade849f5d8baaa55f2d9
Jul 28 22:11:12.820496 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.820473    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e996bb9-7b01-43e8-be2e-5f08f929e82c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p\" (UID: \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"
Jul 28 22:11:12.820602 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.820544    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e996bb9-7b01-43e8-be2e-5f08f929e82c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p\" (UID: \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"
Jul 28 22:11:12.820642 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.820607    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq5v6\" (UniqueName: \"kubernetes.io/projected/5e996bb9-7b01-43e8-be2e-5f08f929e82c-kube-api-access-pq5v6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p\" (UID: \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"
Jul 28 22:11:12.921849 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.921821    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e996bb9-7b01-43e8-be2e-5f08f929e82c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p\" (UID: \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"
Jul 28 22:11:12.921972 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.921908    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq5v6\" (UniqueName: \"kubernetes.io/projected/5e996bb9-7b01-43e8-be2e-5f08f929e82c-kube-api-access-pq5v6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p\" (UID: \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"
Jul 28 22:11:12.921972 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.921948    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e996bb9-7b01-43e8-be2e-5f08f929e82c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p\" (UID: \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"
Jul 28 22:11:12.922197 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.922176    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e996bb9-7b01-43e8-be2e-5f08f929e82c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p\" (UID: \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"
Jul 28 22:11:12.922271 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.922254    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e996bb9-7b01-43e8-be2e-5f08f929e82c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p\" (UID: \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"
Jul 28 22:11:12.930172 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.930143    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq5v6\" (UniqueName: \"kubernetes.io/projected/5e996bb9-7b01-43e8-be2e-5f08f929e82c-kube-api-access-pq5v6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p\" (UID: \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"
Jul 28 22:11:12.994881 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:12.994862    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"
Jul 28 22:11:13.086327 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.086298    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"]
Jul 28 22:11:13.090269 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.090243    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"
Jul 28 22:11:13.098012 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.097775    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"]
Jul 28 22:11:13.124465 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.124442    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"]
Jul 28 22:11:13.126948 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:11:13.126923    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e996bb9_7b01_43e8_be2e_5f08f929e82c.slice/crio-0f71a35dfc4396c59b30299e7dddfa125a133cf4a823ae27cd7ae7f7a6afb8e8 WatchSource:0}: Error finding container 0f71a35dfc4396c59b30299e7dddfa125a133cf4a823ae27cd7ae7f7a6afb8e8: Status 404 returned error can't find the container with id 0f71a35dfc4396c59b30299e7dddfa125a133cf4a823ae27cd7ae7f7a6afb8e8
Jul 28 22:11:13.224740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.224717    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f987892-0ab9-416d-b4e3-1ccee76ef06b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9\" (UID: \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"
Jul 28 22:11:13.224851 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.224747    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92s22\" (UniqueName: \"kubernetes.io/projected/4f987892-0ab9-416d-b4e3-1ccee76ef06b-kube-api-access-92s22\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9\" (UID: \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"
Jul 28 22:11:13.224851 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.224829    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f987892-0ab9-416d-b4e3-1ccee76ef06b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9\" (UID: \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"
Jul 28 22:11:13.325206 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.325172    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f987892-0ab9-416d-b4e3-1ccee76ef06b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9\" (UID: \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"
Jul 28 22:11:13.325619 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.325211    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92s22\" (UniqueName: \"kubernetes.io/projected/4f987892-0ab9-416d-b4e3-1ccee76ef06b-kube-api-access-92s22\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9\" (UID: \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"
Jul 28 22:11:13.325619 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.325251    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f987892-0ab9-416d-b4e3-1ccee76ef06b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9\" (UID: \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"
Jul 28 22:11:13.325619 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.325543    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f987892-0ab9-416d-b4e3-1ccee76ef06b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9\" (UID: \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"
Jul 28 22:11:13.325736 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.325623    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f987892-0ab9-416d-b4e3-1ccee76ef06b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9\" (UID: \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"
Jul 28 22:11:13.334978 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.334957    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92s22\" (UniqueName: \"kubernetes.io/projected/4f987892-0ab9-416d-b4e3-1ccee76ef06b-kube-api-access-92s22\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9\" (UID: \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"
Jul 28 22:11:13.400914 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.400892    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"
Jul 28 22:11:13.416727 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.416700    2583 generic.go:356] "Generic (PLEG): container finished" podID="41654737-66b5-44ea-8bbb-78c923e5af0e" containerID="7da943a52018e68914e064a83cb6c2ea4168e8725625ba1e56f2b0e1656f1157" exitCode=0
Jul 28 22:11:13.416829 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.416761    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn" event={"ID":"41654737-66b5-44ea-8bbb-78c923e5af0e","Type":"ContainerDied","Data":"7da943a52018e68914e064a83cb6c2ea4168e8725625ba1e56f2b0e1656f1157"}
Jul 28 22:11:13.416896 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.416849    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn" event={"ID":"41654737-66b5-44ea-8bbb-78c923e5af0e","Type":"ContainerStarted","Data":"9539f2311803a8e34e960ad8daceab297c2bd4e3b502ade849f5d8baaa55f2d9"}
Jul 28 22:11:13.418472 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.418450    2583 generic.go:356] "Generic (PLEG): container finished" podID="5e996bb9-7b01-43e8-be2e-5f08f929e82c" containerID="c8081d105ca7d6a9998659d6f166b7ca1ecb6ec4179111d7f2e2c13f37d32aa5" exitCode=0
Jul 28 22:11:13.418561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.418529    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p" event={"ID":"5e996bb9-7b01-43e8-be2e-5f08f929e82c","Type":"ContainerDied","Data":"c8081d105ca7d6a9998659d6f166b7ca1ecb6ec4179111d7f2e2c13f37d32aa5"}
Jul 28 22:11:13.418613 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.418574    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p" event={"ID":"5e996bb9-7b01-43e8-be2e-5f08f929e82c","Type":"ContainerStarted","Data":"0f71a35dfc4396c59b30299e7dddfa125a133cf4a823ae27cd7ae7f7a6afb8e8"}
Jul 28 22:11:13.530730 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.530709    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"]
Jul 28 22:11:13.532389 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:11:13.532362    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f987892_0ab9_416d_b4e3_1ccee76ef06b.slice/crio-5fcc0584d09f30c5b4bfe2bdc57413ec47ad0b082ba950d1e94176b491dd5997 WatchSource:0}: Error finding container 5fcc0584d09f30c5b4bfe2bdc57413ec47ad0b082ba950d1e94176b491dd5997: Status 404 returned error can't find the container with id 5fcc0584d09f30c5b4bfe2bdc57413ec47ad0b082ba950d1e94176b491dd5997
Jul 28 22:11:13.679092 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.679067    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"]
Jul 28 22:11:13.682425 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.682410    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"
Jul 28 22:11:13.689804 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.689770    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"]
Jul 28 22:11:13.828853 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.828818    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v\" (UID: \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"
Jul 28 22:11:13.829020 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.828869    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgqv5\" (UniqueName: \"kubernetes.io/projected/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-kube-api-access-pgqv5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v\" (UID: \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"
Jul 28 22:11:13.829020 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.828892    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v\" (UID: \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"
Jul 28 22:11:13.929703 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.929671    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v\" (UID: \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"
Jul 28 22:11:13.929898 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.929717    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgqv5\" (UniqueName: \"kubernetes.io/projected/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-kube-api-access-pgqv5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v\" (UID: \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"
Jul 28 22:11:13.929898 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.929737    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v\" (UID: \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"
Jul 28 22:11:13.930062 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.930043    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v\" (UID: \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"
Jul 28 22:11:13.930102 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.930069    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v\" (UID: \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"
Jul 28 22:11:13.937785 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:13.937759    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgqv5\" (UniqueName: \"kubernetes.io/projected/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-kube-api-access-pgqv5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v\" (UID: \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"
Jul 28 22:11:14.012238 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:14.012164    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"
Jul 28 22:11:14.154854 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:14.154826    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"]
Jul 28 22:11:14.250450 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:11:14.250411    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6945deb3_3cc1_48e1_911a_2ec3e9c5fba2.slice/crio-b89f064d417719ce533babcdf4c533bb18fb759d6964244750b4c1a8421be5ae WatchSource:0}: Error finding container b89f064d417719ce533babcdf4c533bb18fb759d6964244750b4c1a8421be5ae: Status 404 returned error can't find the container with id b89f064d417719ce533babcdf4c533bb18fb759d6964244750b4c1a8421be5ae
Jul 28 22:11:14.425381 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:14.425350    2583 generic.go:356] "Generic (PLEG): container finished" podID="4f987892-0ab9-416d-b4e3-1ccee76ef06b" containerID="3514646f048392019c72c3b8e969cf8988686c6995e04bad2444d8868712ca3b" exitCode=0
Jul 28 22:11:14.427358 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:14.427338    2583 generic.go:356] "Generic (PLEG): container finished" podID="41654737-66b5-44ea-8bbb-78c923e5af0e" containerID="f6e0d2833d46253a4d66d0ef16108768d99b047f489ef138a48ce2fac6852cde" exitCode=0
Jul 28 22:11:14.429319 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:14.429295    2583 generic.go:356] "Generic (PLEG): container finished" podID="5e996bb9-7b01-43e8-be2e-5f08f929e82c" containerID="39d90ac6f8a895c16768fc7f0cc12757ae93ecbe3d6f3eb3172517d59af0c56f" exitCode=0
Jul 28 22:11:14.430988 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:14.430967    2583 generic.go:356] "Generic (PLEG): container finished" podID="6945deb3-3cc1-48e1-911a-2ec3e9c5fba2" containerID="4ed7a5d1e7d14b7a3be2868d64902a203dc276d53440fa0d83519b324f4a1535" exitCode=0
Jul 28 22:11:14.431830 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:14.431784    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9" event={"ID":"4f987892-0ab9-416d-b4e3-1ccee76ef06b","Type":"ContainerDied","Data":"3514646f048392019c72c3b8e969cf8988686c6995e04bad2444d8868712ca3b"}
Jul 28 22:11:14.431937 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:14.431849    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9" event={"ID":"4f987892-0ab9-416d-b4e3-1ccee76ef06b","Type":"ContainerStarted","Data":"5fcc0584d09f30c5b4bfe2bdc57413ec47ad0b082ba950d1e94176b491dd5997"}
Jul 28 22:11:14.431937 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:14.431863    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn" event={"ID":"41654737-66b5-44ea-8bbb-78c923e5af0e","Type":"ContainerDied","Data":"f6e0d2833d46253a4d66d0ef16108768d99b047f489ef138a48ce2fac6852cde"}
Jul 28 22:11:14.431937 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:14.431897    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p" event={"ID":"5e996bb9-7b01-43e8-be2e-5f08f929e82c","Type":"ContainerDied","Data":"39d90ac6f8a895c16768fc7f0cc12757ae93ecbe3d6f3eb3172517d59af0c56f"}
Jul 28 22:11:14.431937 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:14.431914    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v" event={"ID":"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2","Type":"ContainerDied","Data":"4ed7a5d1e7d14b7a3be2868d64902a203dc276d53440fa0d83519b324f4a1535"}
Jul 28 22:11:14.431937 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:14.431930    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v" event={"ID":"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2","Type":"ContainerStarted","Data":"b89f064d417719ce533babcdf4c533bb18fb759d6964244750b4c1a8421be5ae"}
Jul 28 22:11:15.437419 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:15.437383    2583 generic.go:356] "Generic (PLEG): container finished" podID="5e996bb9-7b01-43e8-be2e-5f08f929e82c" containerID="396bf9901c5ec7eece916a4124952f21ea16890cc93d66b143dff44ae296ed03" exitCode=0
Jul 28 22:11:15.437848 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:15.437464    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p" event={"ID":"5e996bb9-7b01-43e8-be2e-5f08f929e82c","Type":"ContainerDied","Data":"396bf9901c5ec7eece916a4124952f21ea16890cc93d66b143dff44ae296ed03"}
Jul 28 22:11:15.439056 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:15.439028    2583 generic.go:356] "Generic (PLEG): container finished" podID="6945deb3-3cc1-48e1-911a-2ec3e9c5fba2" containerID="39de88768cb9c8a7e40563a28170922e7c0eda2e18de5b8a43789c1b57929f0d" exitCode=0
Jul 28 22:11:15.439145 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:15.439108    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v" event={"ID":"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2","Type":"ContainerDied","Data":"39de88768cb9c8a7e40563a28170922e7c0eda2e18de5b8a43789c1b57929f0d"}
Jul 28 22:11:15.441080 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:15.441042    2583 generic.go:356] "Generic (PLEG): container finished" podID="4f987892-0ab9-416d-b4e3-1ccee76ef06b" containerID="79fe0a2c472b88e322dd2d727f2c0f1940f3e271c7651885fec72b260696d932" exitCode=0
Jul 28 22:11:15.441181 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:15.441098    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9" event={"ID":"4f987892-0ab9-416d-b4e3-1ccee76ef06b","Type":"ContainerDied","Data":"79fe0a2c472b88e322dd2d727f2c0f1940f3e271c7651885fec72b260696d932"}
Jul 28 22:11:15.443305 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:15.443286    2583 generic.go:356] "Generic (PLEG): container finished" podID="41654737-66b5-44ea-8bbb-78c923e5af0e" containerID="0d0f405c2b6daa7c3b4b678b736f79f40ef1c1bd4e2edd20ec3503ac66fd6c53" exitCode=0
Jul 28 22:11:15.443388 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:15.443315    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn" event={"ID":"41654737-66b5-44ea-8bbb-78c923e5af0e","Type":"ContainerDied","Data":"0d0f405c2b6daa7c3b4b678b736f79f40ef1c1bd4e2edd20ec3503ac66fd6c53"}
Jul 28 22:11:16.450559 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.450529    2583 generic.go:356] "Generic (PLEG): container finished" podID="4f987892-0ab9-416d-b4e3-1ccee76ef06b" containerID="c49dfe6e43c364510ddb46fb4626e26428f0f9de493e923a19be55829726176b" exitCode=0
Jul 28 22:11:16.450941 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.450608    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9" event={"ID":"4f987892-0ab9-416d-b4e3-1ccee76ef06b","Type":"ContainerDied","Data":"c49dfe6e43c364510ddb46fb4626e26428f0f9de493e923a19be55829726176b"}
Jul 28 22:11:16.452574 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.452542    2583 generic.go:356] "Generic (PLEG): container finished" podID="6945deb3-3cc1-48e1-911a-2ec3e9c5fba2" containerID="cd858ad8df065a7115d4b7c348a206082090ac229f407fed6747491028fc90dc" exitCode=0
Jul 28 22:11:16.452704 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.452618    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v" event={"ID":"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2","Type":"ContainerDied","Data":"cd858ad8df065a7115d4b7c348a206082090ac229f407fed6747491028fc90dc"}
Jul 28 22:11:16.585693 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.585670    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"
Jul 28 22:11:16.614123 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.614098    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"
Jul 28 22:11:16.754627 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.754535    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41654737-66b5-44ea-8bbb-78c923e5af0e-bundle\") pod \"41654737-66b5-44ea-8bbb-78c923e5af0e\" (UID: \"41654737-66b5-44ea-8bbb-78c923e5af0e\") "
Jul 28 22:11:16.754627 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.754588    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq5v6\" (UniqueName: \"kubernetes.io/projected/5e996bb9-7b01-43e8-be2e-5f08f929e82c-kube-api-access-pq5v6\") pod \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\" (UID: \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\") "
Jul 28 22:11:16.754627 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.754618    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41654737-66b5-44ea-8bbb-78c923e5af0e-util\") pod \"41654737-66b5-44ea-8bbb-78c923e5af0e\" (UID: \"41654737-66b5-44ea-8bbb-78c923e5af0e\") "
Jul 28 22:11:16.755003 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.754735    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e996bb9-7b01-43e8-be2e-5f08f929e82c-bundle\") pod \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\" (UID: \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\") "
Jul 28 22:11:16.755003 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.754842    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h66pv\" (UniqueName: \"kubernetes.io/projected/41654737-66b5-44ea-8bbb-78c923e5af0e-kube-api-access-h66pv\") pod \"41654737-66b5-44ea-8bbb-78c923e5af0e\" (UID: \"41654737-66b5-44ea-8bbb-78c923e5af0e\") "
Jul 28 22:11:16.755003 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.754877    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e996bb9-7b01-43e8-be2e-5f08f929e82c-util\") pod \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\" (UID: \"5e996bb9-7b01-43e8-be2e-5f08f929e82c\") "
Jul 28 22:11:16.755345 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.755258    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41654737-66b5-44ea-8bbb-78c923e5af0e-bundle" (OuterVolumeSpecName: "bundle") pod "41654737-66b5-44ea-8bbb-78c923e5af0e" (UID: "41654737-66b5-44ea-8bbb-78c923e5af0e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:11:16.755345 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.755268    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e996bb9-7b01-43e8-be2e-5f08f929e82c-bundle" (OuterVolumeSpecName: "bundle") pod "5e996bb9-7b01-43e8-be2e-5f08f929e82c" (UID: "5e996bb9-7b01-43e8-be2e-5f08f929e82c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:11:16.757481 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.757457    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e996bb9-7b01-43e8-be2e-5f08f929e82c-kube-api-access-pq5v6" (OuterVolumeSpecName: "kube-api-access-pq5v6") pod "5e996bb9-7b01-43e8-be2e-5f08f929e82c" (UID: "5e996bb9-7b01-43e8-be2e-5f08f929e82c"). InnerVolumeSpecName "kube-api-access-pq5v6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:11:16.757558 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.757541    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41654737-66b5-44ea-8bbb-78c923e5af0e-kube-api-access-h66pv" (OuterVolumeSpecName: "kube-api-access-h66pv") pod "41654737-66b5-44ea-8bbb-78c923e5af0e" (UID: "41654737-66b5-44ea-8bbb-78c923e5af0e"). InnerVolumeSpecName "kube-api-access-h66pv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:11:16.763572 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.763545    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e996bb9-7b01-43e8-be2e-5f08f929e82c-util" (OuterVolumeSpecName: "util") pod "5e996bb9-7b01-43e8-be2e-5f08f929e82c" (UID: "5e996bb9-7b01-43e8-be2e-5f08f929e82c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:11:16.763806 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.763777    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41654737-66b5-44ea-8bbb-78c923e5af0e-util" (OuterVolumeSpecName: "util") pod "41654737-66b5-44ea-8bbb-78c923e5af0e" (UID: "41654737-66b5-44ea-8bbb-78c923e5af0e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:11:16.855964 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.855933    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq5v6\" (UniqueName: \"kubernetes.io/projected/5e996bb9-7b01-43e8-be2e-5f08f929e82c-kube-api-access-pq5v6\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:11:16.855964 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.855956    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41654737-66b5-44ea-8bbb-78c923e5af0e-util\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:11:16.855964 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.855966    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e996bb9-7b01-43e8-be2e-5f08f929e82c-bundle\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:11:16.856158 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.855974    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h66pv\" (UniqueName: \"kubernetes.io/projected/41654737-66b5-44ea-8bbb-78c923e5af0e-kube-api-access-h66pv\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:11:16.856158 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.855983    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e996bb9-7b01-43e8-be2e-5f08f929e82c-util\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:11:16.856158 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:16.855991    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41654737-66b5-44ea-8bbb-78c923e5af0e-bundle\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:11:17.460332 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.460305    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn"
Jul 28 22:11:17.460332 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.460310    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn" event={"ID":"41654737-66b5-44ea-8bbb-78c923e5af0e","Type":"ContainerDied","Data":"9539f2311803a8e34e960ad8daceab297c2bd4e3b502ade849f5d8baaa55f2d9"}
Jul 28 22:11:17.460875 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.460350    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9539f2311803a8e34e960ad8daceab297c2bd4e3b502ade849f5d8baaa55f2d9"
Jul 28 22:11:17.462222 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.462197    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p"
Jul 28 22:11:17.462222 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.462223    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p" event={"ID":"5e996bb9-7b01-43e8-be2e-5f08f929e82c","Type":"ContainerDied","Data":"0f71a35dfc4396c59b30299e7dddfa125a133cf4a823ae27cd7ae7f7a6afb8e8"}
Jul 28 22:11:17.462381 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.462247    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f71a35dfc4396c59b30299e7dddfa125a133cf4a823ae27cd7ae7f7a6afb8e8"
Jul 28 22:11:17.617203 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.617176    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"
Jul 28 22:11:17.639010 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.638987    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"
Jul 28 22:11:17.762735 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.762657    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-util\") pod \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\" (UID: \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\") "
Jul 28 22:11:17.762735 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.762700    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92s22\" (UniqueName: \"kubernetes.io/projected/4f987892-0ab9-416d-b4e3-1ccee76ef06b-kube-api-access-92s22\") pod \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\" (UID: \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\") "
Jul 28 22:11:17.763093 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.762762    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f987892-0ab9-416d-b4e3-1ccee76ef06b-bundle\") pod \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\" (UID: \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\") "
Jul 28 22:11:17.763093 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.762778    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgqv5\" (UniqueName: \"kubernetes.io/projected/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-kube-api-access-pgqv5\") pod \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\" (UID: \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\") "
Jul 28 22:11:17.763093 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.762840    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-bundle\") pod \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\" (UID: \"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2\") "
Jul 28 22:11:17.763093 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.762865    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f987892-0ab9-416d-b4e3-1ccee76ef06b-util\") pod \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\" (UID: \"4f987892-0ab9-416d-b4e3-1ccee76ef06b\") "
Jul 28 22:11:17.763505 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.763451    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-bundle" (OuterVolumeSpecName: "bundle") pod "6945deb3-3cc1-48e1-911a-2ec3e9c5fba2" (UID: "6945deb3-3cc1-48e1-911a-2ec3e9c5fba2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:11:17.763607 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.763529    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f987892-0ab9-416d-b4e3-1ccee76ef06b-bundle" (OuterVolumeSpecName: "bundle") pod "4f987892-0ab9-416d-b4e3-1ccee76ef06b" (UID: "4f987892-0ab9-416d-b4e3-1ccee76ef06b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:11:17.764971 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.764943    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-kube-api-access-pgqv5" (OuterVolumeSpecName: "kube-api-access-pgqv5") pod "6945deb3-3cc1-48e1-911a-2ec3e9c5fba2" (UID: "6945deb3-3cc1-48e1-911a-2ec3e9c5fba2"). InnerVolumeSpecName "kube-api-access-pgqv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:11:17.765357 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.765329    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f987892-0ab9-416d-b4e3-1ccee76ef06b-kube-api-access-92s22" (OuterVolumeSpecName: "kube-api-access-92s22") pod "4f987892-0ab9-416d-b4e3-1ccee76ef06b" (UID: "4f987892-0ab9-416d-b4e3-1ccee76ef06b"). InnerVolumeSpecName "kube-api-access-92s22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:11:17.768570 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.768549    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f987892-0ab9-416d-b4e3-1ccee76ef06b-util" (OuterVolumeSpecName: "util") pod "4f987892-0ab9-416d-b4e3-1ccee76ef06b" (UID: "4f987892-0ab9-416d-b4e3-1ccee76ef06b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:11:17.768695 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.768678    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-util" (OuterVolumeSpecName: "util") pod "6945deb3-3cc1-48e1-911a-2ec3e9c5fba2" (UID: "6945deb3-3cc1-48e1-911a-2ec3e9c5fba2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:11:17.863915 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.863877    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f987892-0ab9-416d-b4e3-1ccee76ef06b-bundle\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:11:17.863915 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.863911    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgqv5\" (UniqueName: \"kubernetes.io/projected/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-kube-api-access-pgqv5\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:11:17.863915 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.863922    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-bundle\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:11:17.864145 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.863930    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f987892-0ab9-416d-b4e3-1ccee76ef06b-util\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:11:17.864145 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.863938    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6945deb3-3cc1-48e1-911a-2ec3e9c5fba2-util\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:11:17.864145 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:17.863945    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92s22\" (UniqueName: \"kubernetes.io/projected/4f987892-0ab9-416d-b4e3-1ccee76ef06b-kube-api-access-92s22\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:11:18.468443 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:18.468420    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v"
Jul 28 22:11:18.468443 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:18.468434    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v" event={"ID":"6945deb3-3cc1-48e1-911a-2ec3e9c5fba2","Type":"ContainerDied","Data":"b89f064d417719ce533babcdf4c533bb18fb759d6964244750b4c1a8421be5ae"}
Jul 28 22:11:18.468932 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:18.468479    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b89f064d417719ce533babcdf4c533bb18fb759d6964244750b4c1a8421be5ae"
Jul 28 22:11:18.470262 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:18.470241    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9" event={"ID":"4f987892-0ab9-416d-b4e3-1ccee76ef06b","Type":"ContainerDied","Data":"5fcc0584d09f30c5b4bfe2bdc57413ec47ad0b082ba950d1e94176b491dd5997"}
Jul 28 22:11:18.470395 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:18.470266    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9"
Jul 28 22:11:18.470395 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:18.470279    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5fcc0584d09f30c5b4bfe2bdc57413ec47ad0b082ba950d1e94176b491dd5997"
Jul 28 22:11:19.719752 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:19.719713    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:20.326717 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:20.326688    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:25.145054 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:25.145012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:27.232163 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:27.232129    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:27.736510 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:27.736463    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:28.790533 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:28.790497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:30.196185 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:30.196151    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:32.286436 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:32.286397    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:32.393834 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:32.393768    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:34.169156 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:34.169123    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:35.333139 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:35.333095    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:35.390014 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:35.389985    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:35.730429 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:35.730399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:35.790530 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:35.790497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:39.542139 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:39.542050    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:39.640125 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:39.640090    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:40.078382 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078352    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"]
Jul 28 22:11:40.078703 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078690    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e996bb9-7b01-43e8-be2e-5f08f929e82c" containerName="util"
Jul 28 22:11:40.078770 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078704    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e996bb9-7b01-43e8-be2e-5f08f929e82c" containerName="util"
Jul 28 22:11:40.078770 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078712    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f987892-0ab9-416d-b4e3-1ccee76ef06b" containerName="pull"
Jul 28 22:11:40.078770 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078717    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f987892-0ab9-416d-b4e3-1ccee76ef06b" containerName="pull"
Jul 28 22:11:40.078770 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078725    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e996bb9-7b01-43e8-be2e-5f08f929e82c" containerName="pull"
Jul 28 22:11:40.078770 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078731    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e996bb9-7b01-43e8-be2e-5f08f929e82c" containerName="pull"
Jul 28 22:11:40.078770 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078737    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f987892-0ab9-416d-b4e3-1ccee76ef06b" containerName="extract"
Jul 28 22:11:40.078770 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078742    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f987892-0ab9-416d-b4e3-1ccee76ef06b" containerName="extract"
Jul 28 22:11:40.078770 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078749    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6945deb3-3cc1-48e1-911a-2ec3e9c5fba2" containerName="util"
Jul 28 22:11:40.078770 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078754    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6945deb3-3cc1-48e1-911a-2ec3e9c5fba2" containerName="util"
Jul 28 22:11:40.078770 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078761    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41654737-66b5-44ea-8bbb-78c923e5af0e" containerName="util"
Jul 28 22:11:40.078770 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078767    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="41654737-66b5-44ea-8bbb-78c923e5af0e" containerName="util"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078779    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6945deb3-3cc1-48e1-911a-2ec3e9c5fba2" containerName="pull"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078785    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6945deb3-3cc1-48e1-911a-2ec3e9c5fba2" containerName="pull"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078806    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41654737-66b5-44ea-8bbb-78c923e5af0e" containerName="extract"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078812    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="41654737-66b5-44ea-8bbb-78c923e5af0e" containerName="extract"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078818    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6945deb3-3cc1-48e1-911a-2ec3e9c5fba2" containerName="extract"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078823    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6945deb3-3cc1-48e1-911a-2ec3e9c5fba2" containerName="extract"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078833    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41654737-66b5-44ea-8bbb-78c923e5af0e" containerName="pull"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078838    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="41654737-66b5-44ea-8bbb-78c923e5af0e" containerName="pull"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078847    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f987892-0ab9-416d-b4e3-1ccee76ef06b" containerName="util"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078851    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f987892-0ab9-416d-b4e3-1ccee76ef06b" containerName="util"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078858    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e996bb9-7b01-43e8-be2e-5f08f929e82c" containerName="extract"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078863    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e996bb9-7b01-43e8-be2e-5f08f929e82c" containerName="extract"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078931    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e996bb9-7b01-43e8-be2e-5f08f929e82c" containerName="extract"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078938    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f987892-0ab9-416d-b4e3-1ccee76ef06b" containerName="extract"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078947    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6945deb3-3cc1-48e1-911a-2ec3e9c5fba2" containerName="extract"
Jul 28 22:11:40.079561 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.078954    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="41654737-66b5-44ea-8bbb-78c923e5af0e" containerName="extract"
Jul 28 22:11:40.083533 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.083511    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"
Jul 28 22:11:40.085479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.085456    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 28 22:11:40.085479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.085479    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 28 22:11:40.085651 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.085511    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h7fc5\""
Jul 28 22:11:40.085651 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.085530    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 22:11:40.086104 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.086086    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 22:11:40.092064 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.092039    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"]
Jul 28 22:11:40.092175 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.092138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:40.156219 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.156188    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-brlp8\" (UID: \"37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"
Jul 28 22:11:40.156383 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.156240    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlppx\" (UniqueName: \"kubernetes.io/projected/37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad-kube-api-access-xlppx\") pod \"kuadrant-console-plugin-66b5774985-brlp8\" (UID: \"37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"
Jul 28 22:11:40.156383 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.156360    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-brlp8\" (UID: \"37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"
Jul 28 22:11:40.256853 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.256791    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-brlp8\" (UID: \"37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"
Jul 28 22:11:40.257062 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.256960    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-brlp8\" (UID: \"37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"
Jul 28 22:11:40.257062 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.257014    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlppx\" (UniqueName: \"kubernetes.io/projected/37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad-kube-api-access-xlppx\") pod \"kuadrant-console-plugin-66b5774985-brlp8\" (UID: \"37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"
Jul 28 22:11:40.257195 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:11:40.257085    2583 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 28 22:11:40.257195 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:11:40.257152    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad-plugin-serving-cert podName:37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad nodeName:}" failed. No retries permitted until 2026-07-28 22:11:40.757134442 +0000 UTC m=+444.764999978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-brlp8" (UID: "37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad") : secret "plugin-serving-cert" not found
Jul 28 22:11:40.257545 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.257523    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-brlp8\" (UID: \"37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"
Jul 28 22:11:40.264548 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.264527    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlppx\" (UniqueName: \"kubernetes.io/projected/37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad-kube-api-access-xlppx\") pod \"kuadrant-console-plugin-66b5774985-brlp8\" (UID: \"37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"
Jul 28 22:11:40.330263 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.330177    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:40.760867 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.760833    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-brlp8\" (UID: \"37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"
Jul 28 22:11:40.763180 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.763156    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-brlp8\" (UID: \"37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"
Jul 28 22:11:40.994248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:40.994215    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"
Jul 28 22:11:41.125861 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:41.125836    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8"]
Jul 28 22:11:41.127325 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:11:41.127301    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37eb6299_0b0d_48c3_bb38_e0c6cc5b1bad.slice/crio-de623f1655b61deeba8b6549bf9a1ed2ec91cca840c9edc3eb925c0ae23afedb WatchSource:0}: Error finding container de623f1655b61deeba8b6549bf9a1ed2ec91cca840c9edc3eb925c0ae23afedb: Status 404 returned error can't find the container with id de623f1655b61deeba8b6549bf9a1ed2ec91cca840c9edc3eb925c0ae23afedb
Jul 28 22:11:41.579991 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:41.579951    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8" event={"ID":"37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad","Type":"ContainerStarted","Data":"de623f1655b61deeba8b6549bf9a1ed2ec91cca840c9edc3eb925c0ae23afedb"}
Jul 28 22:11:43.348683 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:43.348605    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:43.417005 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:43.416969    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:50.166946 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:50.166906    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:50.272670 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:50.272605    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:53.933184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:53.933132    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:54.032709 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:54.032674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:54.144786 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:54.144711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:54.253265 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:54.253133    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:55.710067 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:55.710023    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:11:55.810706 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:11:55.810654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:00.863850 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:00.863808    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:00.965597 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:00.965540    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:02.707593 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:02.707548    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8" event={"ID":"37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad","Type":"ContainerStarted","Data":"e8bb8e5822215d98cc8229f13910d83be4ac32d5543fbe913d75718c74158369"}
Jul 28 22:12:02.722885 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:02.722831    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-brlp8" podStartSLOduration=1.8501632639999999 podStartE2EDuration="22.72281316s" totalImagesPullingTime="20.872649896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:11:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:12:02.721694351 +0000 UTC m=+466.729559909" watchObservedRunningTime="2026-07-28 22:12:02.72281316 +0000 UTC m=+466.730678715"
Jul 28 22:12:07.929714 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:07.929674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:08.039025 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:08.038990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:11.917966 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:11.917930    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:11.974595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:11.974563    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:12.527410 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:12.527378    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:12.610756 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:12.610712    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:15.523106 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:15.523065    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:15.620921 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:15.620886    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:20.128412 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:20.128376    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:20.228451 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:20.228408    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:23.630778 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:23.630740    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:23.981226 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:23.981149    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:24.177879 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:24.177843    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:24.243595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:24.243511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:27.525132 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:27.525101    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:27.625755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:27.625725    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:37.030668 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.030627    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-d6cds"]
Jul 28 22:12:37.037408 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.037372    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:37.086916 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.086894    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-d6cds"]
Jul 28 22:12:37.087073 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.087007    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds"
Jul 28 22:12:37.089739 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.089721    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 22:12:37.153982 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.153952    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-d6cds"]
Jul 28 22:12:37.163020 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.162991    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:37.264552 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.264520    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-696dn\" (UniqueName: \"kubernetes.io/projected/6b4d624e-47fb-465d-997e-f38ed55d49d9-kube-api-access-696dn\") pod \"limitador-limitador-64c94fd64f-d6cds\" (UID: \"6b4d624e-47fb-465d-997e-f38ed55d49d9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds"
Jul 28 22:12:37.264697 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.264557    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6b4d624e-47fb-465d-997e-f38ed55d49d9-config-file\") pod \"limitador-limitador-64c94fd64f-d6cds\" (UID: \"6b4d624e-47fb-465d-997e-f38ed55d49d9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds"
Jul 28 22:12:37.271032 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.271006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:37.365788 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.365766    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-696dn\" (UniqueName: \"kubernetes.io/projected/6b4d624e-47fb-465d-997e-f38ed55d49d9-kube-api-access-696dn\") pod \"limitador-limitador-64c94fd64f-d6cds\" (UID: \"6b4d624e-47fb-465d-997e-f38ed55d49d9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds"
Jul 28 22:12:37.365934 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.365813    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6b4d624e-47fb-465d-997e-f38ed55d49d9-config-file\") pod \"limitador-limitador-64c94fd64f-d6cds\" (UID: \"6b4d624e-47fb-465d-997e-f38ed55d49d9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds"
Jul 28 22:12:37.366416 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.366396    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6b4d624e-47fb-465d-997e-f38ed55d49d9-config-file\") pod \"limitador-limitador-64c94fd64f-d6cds\" (UID: \"6b4d624e-47fb-465d-997e-f38ed55d49d9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds"
Jul 28 22:12:37.378308 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.378282    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-696dn\" (UniqueName: \"kubernetes.io/projected/6b4d624e-47fb-465d-997e-f38ed55d49d9-kube-api-access-696dn\") pod \"limitador-limitador-64c94fd64f-d6cds\" (UID: \"6b4d624e-47fb-465d-997e-f38ed55d49d9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds"
Jul 28 22:12:37.400269 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.400247    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds"
Jul 28 22:12:37.534884 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.534860    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-d6cds"]
Jul 28 22:12:37.536754 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:12:37.536729    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b4d624e_47fb_465d_997e_f38ed55d49d9.slice/crio-d4c4ebdf53326edc5065ff31ca37febb9438edf6f84b32b08a87514d01d2ac2a WatchSource:0}: Error finding container d4c4ebdf53326edc5065ff31ca37febb9438edf6f84b32b08a87514d01d2ac2a: Status 404 returned error can't find the container with id d4c4ebdf53326edc5065ff31ca37febb9438edf6f84b32b08a87514d01d2ac2a
Jul 28 22:12:37.887834 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.887789    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds" event={"ID":"6b4d624e-47fb-465d-997e-f38ed55d49d9","Type":"ContainerStarted","Data":"d4c4ebdf53326edc5065ff31ca37febb9438edf6f84b32b08a87514d01d2ac2a"}
Jul 28 22:12:37.987361 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:37.987333    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:38.053991 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:38.053959    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:38.108417 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:38.108383    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-5lw6j"]
Jul 28 22:12:38.111598 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:38.111583    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5lw6j"
Jul 28 22:12:38.113637 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:38.113613    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-75hvs\""
Jul 28 22:12:38.116771 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:38.116753    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5lw6j"]
Jul 28 22:12:38.275246 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:38.275163    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt7dm\" (UniqueName: \"kubernetes.io/projected/4b029e8d-b2ea-4cc3-a17f-f6a553cac289-kube-api-access-mt7dm\") pod \"authorino-75d556bd77-5lw6j\" (UID: \"4b029e8d-b2ea-4cc3-a17f-f6a553cac289\") " pod="kuadrant-system/authorino-75d556bd77-5lw6j"
Jul 28 22:12:38.376869 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:38.376532    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt7dm\" (UniqueName: \"kubernetes.io/projected/4b029e8d-b2ea-4cc3-a17f-f6a553cac289-kube-api-access-mt7dm\") pod \"authorino-75d556bd77-5lw6j\" (UID: \"4b029e8d-b2ea-4cc3-a17f-f6a553cac289\") " pod="kuadrant-system/authorino-75d556bd77-5lw6j"
Jul 28 22:12:38.386422 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:38.386391    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt7dm\" (UniqueName: \"kubernetes.io/projected/4b029e8d-b2ea-4cc3-a17f-f6a553cac289-kube-api-access-mt7dm\") pod \"authorino-75d556bd77-5lw6j\" (UID: \"4b029e8d-b2ea-4cc3-a17f-f6a553cac289\") " pod="kuadrant-system/authorino-75d556bd77-5lw6j"
Jul 28 22:12:38.422080 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:38.422052    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5lw6j"
Jul 28 22:12:38.582911 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:38.582777    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5lw6j"]
Jul 28 22:12:38.585841 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:12:38.585812    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b029e8d_b2ea_4cc3_a17f_f6a553cac289.slice/crio-20f2873076f0c244c286688a89559f2d159504c8cb41d8c4e8f2d5630e323dff WatchSource:0}: Error finding container 20f2873076f0c244c286688a89559f2d159504c8cb41d8c4e8f2d5630e323dff: Status 404 returned error can't find the container with id 20f2873076f0c244c286688a89559f2d159504c8cb41d8c4e8f2d5630e323dff
Jul 28 22:12:38.894496 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:38.894462    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5lw6j" event={"ID":"4b029e8d-b2ea-4cc3-a17f-f6a553cac289","Type":"ContainerStarted","Data":"20f2873076f0c244c286688a89559f2d159504c8cb41d8c4e8f2d5630e323dff"}
Jul 28 22:12:40.908242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:40.908202    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds" event={"ID":"6b4d624e-47fb-465d-997e-f38ed55d49d9","Type":"ContainerStarted","Data":"2ccc3a076df51920f7f29f80469d083ad2d2895d6b2c13b194ba3456298b7faf"}
Jul 28 22:12:40.908630 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:40.908300    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds"
Jul 28 22:12:40.935873 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:40.935822    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds" podStartSLOduration=1.075149823 podStartE2EDuration="3.93580595s" totalImagesPullingTime="2.860656127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:12:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:12:40.933824048 +0000 UTC m=+504.941689630" watchObservedRunningTime="2026-07-28 22:12:40.93580595 +0000 UTC m=+504.943671500"
Jul 28 22:12:42.921409 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:42.921369    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5lw6j" event={"ID":"4b029e8d-b2ea-4cc3-a17f-f6a553cac289","Type":"ContainerStarted","Data":"2010804a8fdcf3cde01ac1819a2217cd292e4b205ba5ef300f170dd739c3f385"}
Jul 28 22:12:42.935321 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:42.935264    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-5lw6j" podStartSLOduration=1.314380882 podStartE2EDuration="4.935245611s" totalImagesPullingTime="3.620864729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:12:42.934120015 +0000 UTC m=+506.941985575" watchObservedRunningTime="2026-07-28 22:12:42.935245611 +0000 UTC m=+506.943111173"
Jul 28 22:12:51.916032 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:51.915999    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds"
Jul 28 22:12:52.389270 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:52.389236    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-d6cds"]
Jul 28 22:12:52.389456 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:52.389407    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds" podUID="6b4d624e-47fb-465d-997e-f38ed55d49d9" containerName="limitador" containerID="cri-o://2ccc3a076df51920f7f29f80469d083ad2d2895d6b2c13b194ba3456298b7faf" gracePeriod=30
Jul 28 22:12:52.927405 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:52.927383    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds"
Jul 28 22:12:52.978232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:52.978149    2583 generic.go:356] "Generic (PLEG): container finished" podID="6b4d624e-47fb-465d-997e-f38ed55d49d9" containerID="2ccc3a076df51920f7f29f80469d083ad2d2895d6b2c13b194ba3456298b7faf" exitCode=0
Jul 28 22:12:52.978232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:52.978205    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds"
Jul 28 22:12:52.978410 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:52.978240    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds" event={"ID":"6b4d624e-47fb-465d-997e-f38ed55d49d9","Type":"ContainerDied","Data":"2ccc3a076df51920f7f29f80469d083ad2d2895d6b2c13b194ba3456298b7faf"}
Jul 28 22:12:52.978410 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:52.978317    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-d6cds" event={"ID":"6b4d624e-47fb-465d-997e-f38ed55d49d9","Type":"ContainerDied","Data":"d4c4ebdf53326edc5065ff31ca37febb9438edf6f84b32b08a87514d01d2ac2a"}
Jul 28 22:12:52.978410 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:52.978345    2583 scope.go:117] "RemoveContainer" containerID="2ccc3a076df51920f7f29f80469d083ad2d2895d6b2c13b194ba3456298b7faf"
Jul 28 22:12:52.986479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:52.986462    2583 scope.go:117] "RemoveContainer" containerID="2ccc3a076df51920f7f29f80469d083ad2d2895d6b2c13b194ba3456298b7faf"
Jul 28 22:12:52.986703 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:12:52.986684    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ccc3a076df51920f7f29f80469d083ad2d2895d6b2c13b194ba3456298b7faf\": container with ID starting with 2ccc3a076df51920f7f29f80469d083ad2d2895d6b2c13b194ba3456298b7faf not found: ID does not exist" containerID="2ccc3a076df51920f7f29f80469d083ad2d2895d6b2c13b194ba3456298b7faf"
Jul 28 22:12:52.986757 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:52.986711    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ccc3a076df51920f7f29f80469d083ad2d2895d6b2c13b194ba3456298b7faf"} err="failed to get container status \"2ccc3a076df51920f7f29f80469d083ad2d2895d6b2c13b194ba3456298b7faf\": rpc error: code = NotFound desc = could not find container \"2ccc3a076df51920f7f29f80469d083ad2d2895d6b2c13b194ba3456298b7faf\": container with ID starting with 2ccc3a076df51920f7f29f80469d083ad2d2895d6b2c13b194ba3456298b7faf not found: ID does not exist"
Jul 28 22:12:52.992133 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:52.992113    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-696dn\" (UniqueName: \"kubernetes.io/projected/6b4d624e-47fb-465d-997e-f38ed55d49d9-kube-api-access-696dn\") pod \"6b4d624e-47fb-465d-997e-f38ed55d49d9\" (UID: \"6b4d624e-47fb-465d-997e-f38ed55d49d9\") "
Jul 28 22:12:52.994037 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:52.994015    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b4d624e-47fb-465d-997e-f38ed55d49d9-kube-api-access-696dn" (OuterVolumeSpecName: "kube-api-access-696dn") pod "6b4d624e-47fb-465d-997e-f38ed55d49d9" (UID: "6b4d624e-47fb-465d-997e-f38ed55d49d9"). InnerVolumeSpecName "kube-api-access-696dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:12:53.093026 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:53.092985    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6b4d624e-47fb-465d-997e-f38ed55d49d9-config-file\") pod \"6b4d624e-47fb-465d-997e-f38ed55d49d9\" (UID: \"6b4d624e-47fb-465d-997e-f38ed55d49d9\") "
Jul 28 22:12:53.093270 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:53.093239    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-696dn\" (UniqueName: \"kubernetes.io/projected/6b4d624e-47fb-465d-997e-f38ed55d49d9-kube-api-access-696dn\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:12:53.093386 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:53.093345    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b4d624e-47fb-465d-997e-f38ed55d49d9-config-file" (OuterVolumeSpecName: "config-file") pod "6b4d624e-47fb-465d-997e-f38ed55d49d9" (UID: "6b4d624e-47fb-465d-997e-f38ed55d49d9"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:12:53.194126 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:53.194091    2583 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6b4d624e-47fb-465d-997e-f38ed55d49d9-config-file\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:12:53.301560 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:53.301521    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-d6cds"]
Jul 28 22:12:53.305492 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:53.305465    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-d6cds"]
Jul 28 22:12:54.431582 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:54.431554    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b4d624e-47fb-465d-997e-f38ed55d49d9" path="/var/lib/kubelet/pods/6b4d624e-47fb-465d-997e-f38ed55d49d9/volumes"
Jul 28 22:12:57.057373 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.057334    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:57.242361 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.242321    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:57.516039 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.516012    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-994mz"]
Jul 28 22:12:57.516321 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.516310    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b4d624e-47fb-465d-997e-f38ed55d49d9" containerName="limitador"
Jul 28 22:12:57.516360 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.516322    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b4d624e-47fb-465d-997e-f38ed55d49d9" containerName="limitador"
Jul 28 22:12:57.516400 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.516391    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b4d624e-47fb-465d-997e-f38ed55d49d9" containerName="limitador"
Jul 28 22:12:57.520711 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.520690    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-994mz"
Jul 28 22:12:57.522856 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.522834    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 28 22:12:57.522960 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.522865    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 22:12:57.523520 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.523504    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 22:12:57.523520 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.523511    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-fnzrn\""
Jul 28 22:12:57.530070 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.530046    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-994mz"]
Jul 28 22:12:57.547550 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.547530    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:12:57.620814 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.620765    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0c4b12f5-75a7-4c7d-987f-499ae220bbe7-data\") pod \"postgres-69fb4594cf-994mz\" (UID: \"0c4b12f5-75a7-4c7d-987f-499ae220bbe7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-994mz"
Jul 28 22:12:57.620928 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.620842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsgtx\" (UniqueName: \"kubernetes.io/projected/0c4b12f5-75a7-4c7d-987f-499ae220bbe7-kube-api-access-dsgtx\") pod \"postgres-69fb4594cf-994mz\" (UID: \"0c4b12f5-75a7-4c7d-987f-499ae220bbe7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-994mz"
Jul 28 22:12:57.722076 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.722045    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0c4b12f5-75a7-4c7d-987f-499ae220bbe7-data\") pod \"postgres-69fb4594cf-994mz\" (UID: \"0c4b12f5-75a7-4c7d-987f-499ae220bbe7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-994mz"
Jul 28 22:12:57.722226 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.722106    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsgtx\" (UniqueName: \"kubernetes.io/projected/0c4b12f5-75a7-4c7d-987f-499ae220bbe7-kube-api-access-dsgtx\") pod \"postgres-69fb4594cf-994mz\" (UID: \"0c4b12f5-75a7-4c7d-987f-499ae220bbe7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-994mz"
Jul 28 22:12:57.722448 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.722428    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0c4b12f5-75a7-4c7d-987f-499ae220bbe7-data\") pod \"postgres-69fb4594cf-994mz\" (UID: \"0c4b12f5-75a7-4c7d-987f-499ae220bbe7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-994mz"
Jul 28 22:12:57.729596 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.729577    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsgtx\" (UniqueName: \"kubernetes.io/projected/0c4b12f5-75a7-4c7d-987f-499ae220bbe7-kube-api-access-dsgtx\") pod \"postgres-69fb4594cf-994mz\" (UID: \"0c4b12f5-75a7-4c7d-987f-499ae220bbe7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-994mz"
Jul 28 22:12:57.832484 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.832395    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-994mz"
Jul 28 22:12:57.964672 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:57.964633    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-994mz"]
Jul 28 22:12:57.972955 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:12:57.972925    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c4b12f5_75a7_4c7d_987f_499ae220bbe7.slice/crio-c9ee94028cc271739965c40ab719d40eb22038f94d67083fe2797a5614b85e11 WatchSource:0}: Error finding container c9ee94028cc271739965c40ab719d40eb22038f94d67083fe2797a5614b85e11: Status 404 returned error can't find the container with id c9ee94028cc271739965c40ab719d40eb22038f94d67083fe2797a5614b85e11
Jul 28 22:12:58.004754 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:12:58.004718    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-994mz" event={"ID":"0c4b12f5-75a7-4c7d-987f-499ae220bbe7","Type":"ContainerStarted","Data":"c9ee94028cc271739965c40ab719d40eb22038f94d67083fe2797a5614b85e11"}
Jul 28 22:13:03.036021 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:03.035982    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-994mz" event={"ID":"0c4b12f5-75a7-4c7d-987f-499ae220bbe7","Type":"ContainerStarted","Data":"6fea0d601da07b5974d1462fe3d2287cf6eb4f2de40838b1e44bf9ae2d66b426"}
Jul 28 22:13:03.036501 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:03.036143    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-994mz"
Jul 28 22:13:03.051517 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:03.051464    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-994mz" podStartSLOduration=1.1158082089999999 podStartE2EDuration="6.051447533s" totalImagesPullingTime="4.935639324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:12:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:13:03.049545744 +0000 UTC m=+527.057411303" watchObservedRunningTime="2026-07-28 22:13:03.051447533 +0000 UTC m=+527.059313092"
Jul 28 22:13:09.070949 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.070879    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-994mz"
Jul 28 22:13:09.478522 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.478483    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:09.636241 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.636208    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t6bkq"]
Jul 28 22:13:09.639552 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.639537    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-t6bkq"
Jul 28 22:13:09.648681 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.648655    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t6bkq"]
Jul 28 22:13:09.710701 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.710668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:09.726214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.726191    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrr9t\" (UniqueName: \"kubernetes.io/projected/8a02ac65-94bb-4690-be79-36d63c387f3e-kube-api-access-wrr9t\") pod \"authorino-5d4b67c446-t6bkq\" (UID: \"8a02ac65-94bb-4690-be79-36d63c387f3e\") " pod="kuadrant-system/authorino-5d4b67c446-t6bkq"
Jul 28 22:13:09.826931 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.826841    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrr9t\" (UniqueName: \"kubernetes.io/projected/8a02ac65-94bb-4690-be79-36d63c387f3e-kube-api-access-wrr9t\") pod \"authorino-5d4b67c446-t6bkq\" (UID: \"8a02ac65-94bb-4690-be79-36d63c387f3e\") " pod="kuadrant-system/authorino-5d4b67c446-t6bkq"
Jul 28 22:13:09.835064 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.835033    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrr9t\" (UniqueName: \"kubernetes.io/projected/8a02ac65-94bb-4690-be79-36d63c387f3e-kube-api-access-wrr9t\") pod \"authorino-5d4b67c446-t6bkq\" (UID: \"8a02ac65-94bb-4690-be79-36d63c387f3e\") " pod="kuadrant-system/authorino-5d4b67c446-t6bkq"
Jul 28 22:13:09.888217 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.888185    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t6bkq"]
Jul 28 22:13:09.888436 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.888425    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-t6bkq"
Jul 28 22:13:09.919473 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.919447    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b6cdbd649-4vrjp"]
Jul 28 22:13:09.923762 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.923744    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6cdbd649-4vrjp"
Jul 28 22:13:09.929278 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.929251    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b6cdbd649-4vrjp"]
Jul 28 22:13:09.985392 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:09.985362    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b6cdbd649-4vrjp"]
Jul 28 22:13:09.985651 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:13:09.985632    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-r5p7t], unattached volumes=[], failed to process volumes=[kube-api-access-r5p7t]: context canceled" pod="kuadrant-system/authorino-b6cdbd649-4vrjp" podUID="9f0a5f4d-d639-45f9-9f7b-6eaee0117d06"
Jul 28 22:13:10.012691 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.012661    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-857f577989-92rr5"]
Jul 28 22:13:10.016124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.016104    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-857f577989-92rr5"
Jul 28 22:13:10.018007 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.017990    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 22:13:10.023642 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.023614    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-857f577989-92rr5"]
Jul 28 22:13:10.028780 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.028761    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t6bkq"]
Jul 28 22:13:10.029009 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.028984    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5p7t\" (UniqueName: \"kubernetes.io/projected/9f0a5f4d-d639-45f9-9f7b-6eaee0117d06-kube-api-access-r5p7t\") pod \"authorino-b6cdbd649-4vrjp\" (UID: \"9f0a5f4d-d639-45f9-9f7b-6eaee0117d06\") " pod="kuadrant-system/authorino-b6cdbd649-4vrjp"
Jul 28 22:13:10.030741 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:13:10.030720    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a02ac65_94bb_4690_be79_36d63c387f3e.slice/crio-6a309521a17d0b6b740c1c55564f7defe7a76d3bd59bcc7f478b2bf964758641 WatchSource:0}: Error finding container 6a309521a17d0b6b740c1c55564f7defe7a76d3bd59bcc7f478b2bf964758641: Status 404 returned error can't find the container with id 6a309521a17d0b6b740c1c55564f7defe7a76d3bd59bcc7f478b2bf964758641
Jul 28 22:13:10.074377 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.074348    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6cdbd649-4vrjp"
Jul 28 22:13:10.074743 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.074372    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-t6bkq" event={"ID":"8a02ac65-94bb-4690-be79-36d63c387f3e","Type":"ContainerStarted","Data":"6a309521a17d0b6b740c1c55564f7defe7a76d3bd59bcc7f478b2bf964758641"}
Jul 28 22:13:10.081441 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.081392    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6cdbd649-4vrjp"
Jul 28 22:13:10.130119 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.130086    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5p7t\" (UniqueName: \"kubernetes.io/projected/9f0a5f4d-d639-45f9-9f7b-6eaee0117d06-kube-api-access-r5p7t\") pod \"authorino-b6cdbd649-4vrjp\" (UID: \"9f0a5f4d-d639-45f9-9f7b-6eaee0117d06\") " pod="kuadrant-system/authorino-b6cdbd649-4vrjp"
Jul 28 22:13:10.130119 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.130120    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9b9d945a-cdb7-4d6c-8f24-c2811acfefaf-tls-cert\") pod \"authorino-857f577989-92rr5\" (UID: \"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf\") " pod="kuadrant-system/authorino-857f577989-92rr5"
Jul 28 22:13:10.130301 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.130166    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66n9h\" (UniqueName: \"kubernetes.io/projected/9b9d945a-cdb7-4d6c-8f24-c2811acfefaf-kube-api-access-66n9h\") pod \"authorino-857f577989-92rr5\" (UID: \"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf\") " pod="kuadrant-system/authorino-857f577989-92rr5"
Jul 28 22:13:10.137809 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.137774    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5p7t\" (UniqueName: \"kubernetes.io/projected/9f0a5f4d-d639-45f9-9f7b-6eaee0117d06-kube-api-access-r5p7t\") pod \"authorino-b6cdbd649-4vrjp\" (UID: \"9f0a5f4d-d639-45f9-9f7b-6eaee0117d06\") " pod="kuadrant-system/authorino-b6cdbd649-4vrjp"
Jul 28 22:13:10.230979 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.230950    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5p7t\" (UniqueName: \"kubernetes.io/projected/9f0a5f4d-d639-45f9-9f7b-6eaee0117d06-kube-api-access-r5p7t\") pod \"9f0a5f4d-d639-45f9-9f7b-6eaee0117d06\" (UID: \"9f0a5f4d-d639-45f9-9f7b-6eaee0117d06\") "
Jul 28 22:13:10.231103 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.231044    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9b9d945a-cdb7-4d6c-8f24-c2811acfefaf-tls-cert\") pod \"authorino-857f577989-92rr5\" (UID: \"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf\") " pod="kuadrant-system/authorino-857f577989-92rr5"
Jul 28 22:13:10.231103 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.231083    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66n9h\" (UniqueName: \"kubernetes.io/projected/9b9d945a-cdb7-4d6c-8f24-c2811acfefaf-kube-api-access-66n9h\") pod \"authorino-857f577989-92rr5\" (UID: \"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf\") " pod="kuadrant-system/authorino-857f577989-92rr5"
Jul 28 22:13:10.232975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.232943    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f0a5f4d-d639-45f9-9f7b-6eaee0117d06-kube-api-access-r5p7t" (OuterVolumeSpecName: "kube-api-access-r5p7t") pod "9f0a5f4d-d639-45f9-9f7b-6eaee0117d06" (UID: "9f0a5f4d-d639-45f9-9f7b-6eaee0117d06"). InnerVolumeSpecName "kube-api-access-r5p7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:13:10.233347 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.233329    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9b9d945a-cdb7-4d6c-8f24-c2811acfefaf-tls-cert\") pod \"authorino-857f577989-92rr5\" (UID: \"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf\") " pod="kuadrant-system/authorino-857f577989-92rr5"
Jul 28 22:13:10.238343 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.238324    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66n9h\" (UniqueName: \"kubernetes.io/projected/9b9d945a-cdb7-4d6c-8f24-c2811acfefaf-kube-api-access-66n9h\") pod \"authorino-857f577989-92rr5\" (UID: \"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf\") " pod="kuadrant-system/authorino-857f577989-92rr5"
Jul 28 22:13:10.325670 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.325636    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-857f577989-92rr5"
Jul 28 22:13:10.332179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.332133    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5p7t\" (UniqueName: \"kubernetes.io/projected/9f0a5f4d-d639-45f9-9f7b-6eaee0117d06-kube-api-access-r5p7t\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:13:10.467233 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:10.467211    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-857f577989-92rr5"]
Jul 28 22:13:10.469087 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:13:10.469065    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b9d945a_cdb7_4d6c_8f24_c2811acfefaf.slice/crio-1846e0c62d0bcc51090c9ae60b49656b2c9373ab3d7630dc19357d9b6f864615 WatchSource:0}: Error finding container 1846e0c62d0bcc51090c9ae60b49656b2c9373ab3d7630dc19357d9b6f864615: Status 404 returned error can't find the container with id 1846e0c62d0bcc51090c9ae60b49656b2c9373ab3d7630dc19357d9b6f864615
Jul 28 22:13:11.080872 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.080758    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-857f577989-92rr5" event={"ID":"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf","Type":"ContainerStarted","Data":"d60c4879d39e2f5fdb8436f3411eb29c367fe688bb667a447334cb7aed8f7a2d"}
Jul 28 22:13:11.080872 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.080828    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-857f577989-92rr5" event={"ID":"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf","Type":"ContainerStarted","Data":"1846e0c62d0bcc51090c9ae60b49656b2c9373ab3d7630dc19357d9b6f864615"}
Jul 28 22:13:11.082154 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.082130    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-t6bkq" event={"ID":"8a02ac65-94bb-4690-be79-36d63c387f3e","Type":"ContainerStarted","Data":"1acd032392df36c64485aad29893f76c5a6b1a507440477657827f5f6680e518"}
Jul 28 22:13:11.082154 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.082147    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6cdbd649-4vrjp"
Jul 28 22:13:11.082308 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.082205    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-t6bkq" podUID="8a02ac65-94bb-4690-be79-36d63c387f3e" containerName="authorino" containerID="cri-o://1acd032392df36c64485aad29893f76c5a6b1a507440477657827f5f6680e518" gracePeriod=30
Jul 28 22:13:11.099628 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.099589    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-857f577989-92rr5" podStartSLOduration=1.750274176 podStartE2EDuration="2.099576272s" totalImagesPullingTime="349.302096ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:13:11.097182503 +0000 UTC m=+535.105048061" watchObservedRunningTime="2026-07-28 22:13:11.099576272 +0000 UTC m=+535.107441830"
Jul 28 22:13:11.115407 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.115355    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-t6bkq" podStartSLOduration=1.68368265 podStartE2EDuration="2.115335929s" totalImagesPullingTime="431.653279ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:13:11.113168422 +0000 UTC m=+535.121033981" watchObservedRunningTime="2026-07-28 22:13:11.115335929 +0000 UTC m=+535.123201488"
Jul 28 22:13:11.138340 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.138310    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5lw6j"]
Jul 28 22:13:11.138505 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.138485    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-5lw6j" podUID="4b029e8d-b2ea-4cc3-a17f-f6a553cac289" containerName="authorino" containerID="cri-o://2010804a8fdcf3cde01ac1819a2217cd292e4b205ba5ef300f170dd739c3f385" gracePeriod=30
Jul 28 22:13:11.144484 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.144470    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b6cdbd649-4vrjp"]
Jul 28 22:13:11.152146 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.152119    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b6cdbd649-4vrjp"]
Jul 28 22:13:11.397357 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.397334    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-t6bkq"
Jul 28 22:13:11.400007 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.399988    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5lw6j"
Jul 28 22:13:11.443691 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.443582    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrr9t\" (UniqueName: \"kubernetes.io/projected/8a02ac65-94bb-4690-be79-36d63c387f3e-kube-api-access-wrr9t\") pod \"8a02ac65-94bb-4690-be79-36d63c387f3e\" (UID: \"8a02ac65-94bb-4690-be79-36d63c387f3e\") "
Jul 28 22:13:11.443887 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.443742    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt7dm\" (UniqueName: \"kubernetes.io/projected/4b029e8d-b2ea-4cc3-a17f-f6a553cac289-kube-api-access-mt7dm\") pod \"4b029e8d-b2ea-4cc3-a17f-f6a553cac289\" (UID: \"4b029e8d-b2ea-4cc3-a17f-f6a553cac289\") "
Jul 28 22:13:11.445650 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.445624    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a02ac65-94bb-4690-be79-36d63c387f3e-kube-api-access-wrr9t" (OuterVolumeSpecName: "kube-api-access-wrr9t") pod "8a02ac65-94bb-4690-be79-36d63c387f3e" (UID: "8a02ac65-94bb-4690-be79-36d63c387f3e"). InnerVolumeSpecName "kube-api-access-wrr9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:13:11.445650 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.445638    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b029e8d-b2ea-4cc3-a17f-f6a553cac289-kube-api-access-mt7dm" (OuterVolumeSpecName: "kube-api-access-mt7dm") pod "4b029e8d-b2ea-4cc3-a17f-f6a553cac289" (UID: "4b029e8d-b2ea-4cc3-a17f-f6a553cac289"). InnerVolumeSpecName "kube-api-access-mt7dm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:13:11.544785 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.544749    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt7dm\" (UniqueName: \"kubernetes.io/projected/4b029e8d-b2ea-4cc3-a17f-f6a553cac289-kube-api-access-mt7dm\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:13:11.544785 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:11.544783    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrr9t\" (UniqueName: \"kubernetes.io/projected/8a02ac65-94bb-4690-be79-36d63c387f3e-kube-api-access-wrr9t\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:13:12.088433 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.088395    2583 generic.go:356] "Generic (PLEG): container finished" podID="4b029e8d-b2ea-4cc3-a17f-f6a553cac289" containerID="2010804a8fdcf3cde01ac1819a2217cd292e4b205ba5ef300f170dd739c3f385" exitCode=0
Jul 28 22:13:12.088883 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.088437    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5lw6j"
Jul 28 22:13:12.088883 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.088470    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5lw6j" event={"ID":"4b029e8d-b2ea-4cc3-a17f-f6a553cac289","Type":"ContainerDied","Data":"2010804a8fdcf3cde01ac1819a2217cd292e4b205ba5ef300f170dd739c3f385"}
Jul 28 22:13:12.088883 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.088514    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5lw6j" event={"ID":"4b029e8d-b2ea-4cc3-a17f-f6a553cac289","Type":"ContainerDied","Data":"20f2873076f0c244c286688a89559f2d159504c8cb41d8c4e8f2d5630e323dff"}
Jul 28 22:13:12.088883 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.088531    2583 scope.go:117] "RemoveContainer" containerID="2010804a8fdcf3cde01ac1819a2217cd292e4b205ba5ef300f170dd739c3f385"
Jul 28 22:13:12.090010 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.089991    2583 generic.go:356] "Generic (PLEG): container finished" podID="8a02ac65-94bb-4690-be79-36d63c387f3e" containerID="1acd032392df36c64485aad29893f76c5a6b1a507440477657827f5f6680e518" exitCode=0
Jul 28 22:13:12.090100 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.090034    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-t6bkq"
Jul 28 22:13:12.090100 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.090082    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-t6bkq" event={"ID":"8a02ac65-94bb-4690-be79-36d63c387f3e","Type":"ContainerDied","Data":"1acd032392df36c64485aad29893f76c5a6b1a507440477657827f5f6680e518"}
Jul 28 22:13:12.090196 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.090119    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-t6bkq" event={"ID":"8a02ac65-94bb-4690-be79-36d63c387f3e","Type":"ContainerDied","Data":"6a309521a17d0b6b740c1c55564f7defe7a76d3bd59bcc7f478b2bf964758641"}
Jul 28 22:13:12.097689 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.097672    2583 scope.go:117] "RemoveContainer" containerID="2010804a8fdcf3cde01ac1819a2217cd292e4b205ba5ef300f170dd739c3f385"
Jul 28 22:13:12.097985 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:13:12.097964    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2010804a8fdcf3cde01ac1819a2217cd292e4b205ba5ef300f170dd739c3f385\": container with ID starting with 2010804a8fdcf3cde01ac1819a2217cd292e4b205ba5ef300f170dd739c3f385 not found: ID does not exist" containerID="2010804a8fdcf3cde01ac1819a2217cd292e4b205ba5ef300f170dd739c3f385"
Jul 28 22:13:12.098049 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.097999    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2010804a8fdcf3cde01ac1819a2217cd292e4b205ba5ef300f170dd739c3f385"} err="failed to get container status \"2010804a8fdcf3cde01ac1819a2217cd292e4b205ba5ef300f170dd739c3f385\": rpc error: code = NotFound desc = could not find container \"2010804a8fdcf3cde01ac1819a2217cd292e4b205ba5ef300f170dd739c3f385\": container with ID starting with 2010804a8fdcf3cde01ac1819a2217cd292e4b205ba5ef300f170dd739c3f385 not found: ID does not exist"
Jul 28 22:13:12.098049 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.098022    2583 scope.go:117] "RemoveContainer" containerID="1acd032392df36c64485aad29893f76c5a6b1a507440477657827f5f6680e518"
Jul 28 22:13:12.106061 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.106038    2583 scope.go:117] "RemoveContainer" containerID="1acd032392df36c64485aad29893f76c5a6b1a507440477657827f5f6680e518"
Jul 28 22:13:12.106307 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:13:12.106289    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1acd032392df36c64485aad29893f76c5a6b1a507440477657827f5f6680e518\": container with ID starting with 1acd032392df36c64485aad29893f76c5a6b1a507440477657827f5f6680e518 not found: ID does not exist" containerID="1acd032392df36c64485aad29893f76c5a6b1a507440477657827f5f6680e518"
Jul 28 22:13:12.106373 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.106316    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1acd032392df36c64485aad29893f76c5a6b1a507440477657827f5f6680e518"} err="failed to get container status \"1acd032392df36c64485aad29893f76c5a6b1a507440477657827f5f6680e518\": rpc error: code = NotFound desc = could not find container \"1acd032392df36c64485aad29893f76c5a6b1a507440477657827f5f6680e518\": container with ID starting with 1acd032392df36c64485aad29893f76c5a6b1a507440477657827f5f6680e518 not found: ID does not exist"
Jul 28 22:13:12.118629 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.118603    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5lw6j"]
Jul 28 22:13:12.125105 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.125086    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5lw6j"]
Jul 28 22:13:12.136020 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.136003    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t6bkq"]
Jul 28 22:13:12.139915 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.139891    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t6bkq"]
Jul 28 22:13:12.431476 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.431442    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b029e8d-b2ea-4cc3-a17f-f6a553cac289" path="/var/lib/kubelet/pods/4b029e8d-b2ea-4cc3-a17f-f6a553cac289/volumes"
Jul 28 22:13:12.431781 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.431768    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a02ac65-94bb-4690-be79-36d63c387f3e" path="/var/lib/kubelet/pods/8a02ac65-94bb-4690-be79-36d63c387f3e/volumes"
Jul 28 22:13:12.432062 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:12.432050    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f0a5f4d-d639-45f9-9f7b-6eaee0117d06" path="/var/lib/kubelet/pods/9f0a5f4d-d639-45f9-9f7b-6eaee0117d06/volumes"
Jul 28 22:13:16.000448 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:16.000415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:16.093604 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:16.093569    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:16.151409 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:16.151375    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:19.092266 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:19.092213    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:19.189479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:19.189439    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:31.131051 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:31.131010    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:45.110270 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:45.110232    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:45.190590 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:45.190553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:47.193821 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:47.193745    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:47.295343 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:47.295311    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:50.852495 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:50.852457    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:50.947772 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:50.947738    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:51.440330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:51.440288    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:51.627861 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:51.627828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:52.049341 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:52.049303    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:52.145263 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:52.145223    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:59.546643 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:59.546610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:13:59.647985 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:13:59.647942    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:14:01.810339 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:01.810307    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:14:01.911513 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:01.911484    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:14:12.880200 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.880170    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"]
Jul 28 22:14:12.880552 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.880504    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b029e8d-b2ea-4cc3-a17f-f6a553cac289" containerName="authorino"
Jul 28 22:14:12.880552 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.880516    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b029e8d-b2ea-4cc3-a17f-f6a553cac289" containerName="authorino"
Jul 28 22:14:12.880552 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.880526    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a02ac65-94bb-4690-be79-36d63c387f3e" containerName="authorino"
Jul 28 22:14:12.880552 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.880531    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a02ac65-94bb-4690-be79-36d63c387f3e" containerName="authorino"
Jul 28 22:14:12.880691 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.880592    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b029e8d-b2ea-4cc3-a17f-f6a553cac289" containerName="authorino"
Jul 28 22:14:12.880691 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.880601    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a02ac65-94bb-4690-be79-36d63c387f3e" containerName="authorino"
Jul 28 22:14:12.890518 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.890476    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:12.892739 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.892716    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 22:14:12.892892 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.892763    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 22:14:12.892892 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.892873    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"]
Jul 28 22:14:12.893534 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.893511    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 28 22:14:12.893646 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.893545    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2wf7s\""
Jul 28 22:14:12.912543 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.912512    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:14:12.956071 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.956047    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1679b1b-8fff-4f18-9f3e-42438838654c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:12.956184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.956095    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1679b1b-8fff-4f18-9f3e-42438838654c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:12.956184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.956125    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94vhb\" (UniqueName: \"kubernetes.io/projected/a1679b1b-8fff-4f18-9f3e-42438838654c-kube-api-access-94vhb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:12.956184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.956171    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1679b1b-8fff-4f18-9f3e-42438838654c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:12.956285 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.956217    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1679b1b-8fff-4f18-9f3e-42438838654c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:12.956285 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:12.956242    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1679b1b-8fff-4f18-9f3e-42438838654c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.057418 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.057375    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1679b1b-8fff-4f18-9f3e-42438838654c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.057418 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.057420    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94vhb\" (UniqueName: \"kubernetes.io/projected/a1679b1b-8fff-4f18-9f3e-42438838654c-kube-api-access-94vhb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.057575 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.057544    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1679b1b-8fff-4f18-9f3e-42438838654c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.057611 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.057593    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1679b1b-8fff-4f18-9f3e-42438838654c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.057648 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.057625    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1679b1b-8fff-4f18-9f3e-42438838654c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.057690 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.057677    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1679b1b-8fff-4f18-9f3e-42438838654c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.057912 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.057890    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1679b1b-8fff-4f18-9f3e-42438838654c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.057964 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.057926    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1679b1b-8fff-4f18-9f3e-42438838654c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.057964 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.057950    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1679b1b-8fff-4f18-9f3e-42438838654c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.059834 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.059810    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1679b1b-8fff-4f18-9f3e-42438838654c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.060021 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.060005    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1679b1b-8fff-4f18-9f3e-42438838654c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.064657 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.064642    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94vhb\" (UniqueName: \"kubernetes.io/projected/a1679b1b-8fff-4f18-9f3e-42438838654c-kube-api-access-94vhb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs\" (UID: \"a1679b1b-8fff-4f18-9f3e-42438838654c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.201999 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.201925    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:13.330968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.330939    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"]
Jul 28 22:14:13.333167 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:14:13.333139    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1679b1b_8fff_4f18_9f3e_42438838654c.slice/crio-be90701ecfcdb6f568544e3a299d2b80ddbbe0762e3b8c35ffa06e4f38628834 WatchSource:0}: Error finding container be90701ecfcdb6f568544e3a299d2b80ddbbe0762e3b8c35ffa06e4f38628834: Status 404 returned error can't find the container with id be90701ecfcdb6f568544e3a299d2b80ddbbe0762e3b8c35ffa06e4f38628834
Jul 28 22:14:13.413594 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:13.413563    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs" event={"ID":"a1679b1b-8fff-4f18-9f3e-42438838654c","Type":"ContainerStarted","Data":"be90701ecfcdb6f568544e3a299d2b80ddbbe0762e3b8c35ffa06e4f38628834"}
Jul 28 22:14:15.428438 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:15.428406    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs" event={"ID":"a1679b1b-8fff-4f18-9f3e-42438838654c","Type":"ContainerStarted","Data":"c3ba8495b973e831f57a4a8ff39a9b0eacd2b9a19ba1c529ebce6b08ee5399df"}
Jul 28 22:14:15.443289 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:15.443103    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs" podStartSLOduration=2.191541382 podStartE2EDuration="3.443089459s" totalImagesPullingTime="1.251548077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:14:15.442970996 +0000 UTC m=+599.450836575" watchObservedRunningTime="2026-07-28 22:14:15.443089459 +0000 UTC m=+599.450955018"
Jul 28 22:14:16.365521 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:16.365493    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:14:16.365694 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:16.365661    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:14:20.225257 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:20.225223    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:14:23.202919 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:23.202874    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:23.203467 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:23.202955    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:23.216486 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:23.216464    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:23.478569 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:23.478494    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs"
Jul 28 22:14:32.025890 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:32.025836    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:14:34.282344 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.282307    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"]
Jul 28 22:14:34.286562 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.286533    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.288962 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.288933    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 28 22:14:34.301010 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.300971    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"]
Jul 28 22:14:34.336696 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.336658    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:14:34.438594 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.438564    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77bf9758-8572-4c2d-9293-3f525ce7778e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.438762 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.438623    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77bf9758-8572-4c2d-9293-3f525ce7778e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.438762 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.438653    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77bf9758-8572-4c2d-9293-3f525ce7778e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.438762 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.438675    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77bf9758-8572-4c2d-9293-3f525ce7778e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.438762 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.438706    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jxm9\" (UniqueName: \"kubernetes.io/projected/77bf9758-8572-4c2d-9293-3f525ce7778e-kube-api-access-2jxm9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.438762 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.438724    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77bf9758-8572-4c2d-9293-3f525ce7778e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.540121 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.540047    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77bf9758-8572-4c2d-9293-3f525ce7778e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.540121 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.540085    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77bf9758-8572-4c2d-9293-3f525ce7778e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.540121 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.540115    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77bf9758-8572-4c2d-9293-3f525ce7778e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.540634 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.540596    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxm9\" (UniqueName: \"kubernetes.io/projected/77bf9758-8572-4c2d-9293-3f525ce7778e-kube-api-access-2jxm9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.540762 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.540631    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77bf9758-8572-4c2d-9293-3f525ce7778e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.540762 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.540658    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77bf9758-8572-4c2d-9293-3f525ce7778e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.540762 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.540706    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77bf9758-8572-4c2d-9293-3f525ce7778e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.540762 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.540717    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77bf9758-8572-4c2d-9293-3f525ce7778e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.541068 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.541049    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77bf9758-8572-4c2d-9293-3f525ce7778e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.542453 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.542431    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77bf9758-8572-4c2d-9293-3f525ce7778e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.542912 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.542896    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77bf9758-8572-4c2d-9293-3f525ce7778e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.548132 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.548111    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jxm9\" (UniqueName: \"kubernetes.io/projected/77bf9758-8572-4c2d-9293-3f525ce7778e-kube-api-access-2jxm9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs\" (UID: \"77bf9758-8572-4c2d-9293-3f525ce7778e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.600693 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.600663    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:34.726942 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.726915    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"]
Jul 28 22:14:34.728874 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:14:34.728848    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77bf9758_8572_4c2d_9293_3f525ce7778e.slice/crio-24c80d924b302ffb2ff7744f046a5e1266e6ad4b483910394cc1cb58ab6a21c1 WatchSource:0}: Error finding container 24c80d924b302ffb2ff7744f046a5e1266e6ad4b483910394cc1cb58ab6a21c1: Status 404 returned error can't find the container with id 24c80d924b302ffb2ff7744f046a5e1266e6ad4b483910394cc1cb58ab6a21c1
Jul 28 22:14:34.730224 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:34.730207    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:14:35.527241 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:35.527204    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs" event={"ID":"77bf9758-8572-4c2d-9293-3f525ce7778e","Type":"ContainerStarted","Data":"3a9af67cdcac0200cb7de82e183d6377a61c5337602fd2ca78ab96e954891aa0"}
Jul 28 22:14:35.527615 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:35.527260    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs" event={"ID":"77bf9758-8572-4c2d-9293-3f525ce7778e","Type":"ContainerStarted","Data":"24c80d924b302ffb2ff7744f046a5e1266e6ad4b483910394cc1cb58ab6a21c1"}
Jul 28 22:14:35.544163 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:35.544119    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs" podStartSLOduration=1.3356277699999999 podStartE2EDuration="1.544105094s" totalImagesPullingTime="208.477324ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:14:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:14:35.542071928 +0000 UTC m=+619.549937485" watchObservedRunningTime="2026-07-28 22:14:35.544105094 +0000 UTC m=+619.551970731"
Jul 28 22:14:44.600995 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:44.600917    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:44.600995 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:44.600970    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:44.613000 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:44.612978    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:45.593768 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:45.593741    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs"
Jul 28 22:14:51.419268 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:51.419220    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:14:51.921206 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:51.921167    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:14:52.560084 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:14:52.560045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:15:00.140490 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:00.140451    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl"]
Jul 28 22:15:00.145294 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:00.145276    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl"
Jul 28 22:15:00.147474 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:00.147452    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:15:00.150147 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:00.150119    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl"]
Jul 28 22:15:00.157253 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:00.157231    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl"
Jul 28 22:15:00.284349 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:00.284322    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl"]
Jul 28 22:15:00.286257 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:15:00.286234    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4277813_1e72_4863_8eda_6e5fd847b40d.slice/crio-9f72a4bfff96aa11b4bf0f69d3d4e0f666dec0a76d22451d178ac7b0ffb87b59 WatchSource:0}: Error finding container 9f72a4bfff96aa11b4bf0f69d3d4e0f666dec0a76d22451d178ac7b0ffb87b59: Status 404 returned error can't find the container with id 9f72a4bfff96aa11b4bf0f69d3d4e0f666dec0a76d22451d178ac7b0ffb87b59
Jul 28 22:15:00.657813 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:00.657761    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl" event={"ID":"d4277813-1e72-4863-8eda-6e5fd847b40d","Type":"ContainerStarted","Data":"9f72a4bfff96aa11b4bf0f69d3d4e0f666dec0a76d22451d178ac7b0ffb87b59"}
Jul 28 22:15:03.678621 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:03.678562    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl" event={"ID":"d4277813-1e72-4863-8eda-6e5fd847b40d","Type":"ContainerStarted","Data":"ba8adadc19f171b4184611618373ed621f4b667aedb5b4753d1018f49d58162b"}
Jul 28 22:15:03.694820 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:03.694759    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl" podStartSLOduration=0.623998695 podStartE2EDuration="3.694746456s" totalImagesPullingTime="3.070747761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:15:03.693154797 +0000 UTC m=+647.701020355" watchObservedRunningTime="2026-07-28 22:15:03.694746456 +0000 UTC m=+647.702612011"
Jul 28 22:15:04.684965 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:04.684930    2583 generic.go:356] "Generic (PLEG): container finished" podID="d4277813-1e72-4863-8eda-6e5fd847b40d" containerID="ba8adadc19f171b4184611618373ed621f4b667aedb5b4753d1018f49d58162b" exitCode=0
Jul 28 22:15:04.685341 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:04.685003    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl" event={"ID":"d4277813-1e72-4863-8eda-6e5fd847b40d","Type":"ContainerDied","Data":"ba8adadc19f171b4184611618373ed621f4b667aedb5b4753d1018f49d58162b"}
Jul 28 22:15:05.821393 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:05.821368    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl"
Jul 28 22:15:05.896735 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:05.896706    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:15:06.696473 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:06.696388    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl"
Jul 28 22:15:06.696473 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:06.696407    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl" event={"ID":"d4277813-1e72-4863-8eda-6e5fd847b40d","Type":"ContainerDied","Data":"9f72a4bfff96aa11b4bf0f69d3d4e0f666dec0a76d22451d178ac7b0ffb87b59"}
Jul 28 22:15:06.696473 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:06.696453    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f72a4bfff96aa11b4bf0f69d3d4e0f666dec0a76d22451d178ac7b0ffb87b59"
Jul 28 22:15:18.355326 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:18.355280    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:15:32.996920 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:32.996891    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:15:33.090045 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:15:33.090014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:16:08.501501 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:08.501421    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-857f577989-92rr5"]
Jul 28 22:16:08.501977 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:08.501655    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-857f577989-92rr5" podUID="9b9d945a-cdb7-4d6c-8f24-c2811acfefaf" containerName="authorino" containerID="cri-o://d60c4879d39e2f5fdb8436f3411eb29c367fe688bb667a447334cb7aed8f7a2d" gracePeriod=30
Jul 28 22:16:08.752839 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:08.752762    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-857f577989-92rr5"
Jul 28 22:16:08.833014 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:08.832986    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66n9h\" (UniqueName: \"kubernetes.io/projected/9b9d945a-cdb7-4d6c-8f24-c2811acfefaf-kube-api-access-66n9h\") pod \"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf\" (UID: \"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf\") "
Jul 28 22:16:08.833166 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:08.833054    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9b9d945a-cdb7-4d6c-8f24-c2811acfefaf-tls-cert\") pod \"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf\" (UID: \"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf\") "
Jul 28 22:16:08.834916 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:08.834891    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b9d945a-cdb7-4d6c-8f24-c2811acfefaf-kube-api-access-66n9h" (OuterVolumeSpecName: "kube-api-access-66n9h") pod "9b9d945a-cdb7-4d6c-8f24-c2811acfefaf" (UID: "9b9d945a-cdb7-4d6c-8f24-c2811acfefaf"). InnerVolumeSpecName "kube-api-access-66n9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:16:08.842834 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:08.842812    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b9d945a-cdb7-4d6c-8f24-c2811acfefaf-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9b9d945a-cdb7-4d6c-8f24-c2811acfefaf" (UID: "9b9d945a-cdb7-4d6c-8f24-c2811acfefaf"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:16:08.934313 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:08.934284    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66n9h\" (UniqueName: \"kubernetes.io/projected/9b9d945a-cdb7-4d6c-8f24-c2811acfefaf-kube-api-access-66n9h\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:16:08.934313 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:08.934308    2583 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9b9d945a-cdb7-4d6c-8f24-c2811acfefaf-tls-cert\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:16:09.023605 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:09.023514    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:16:09.028413 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:09.028387    2583 generic.go:356] "Generic (PLEG): container finished" podID="9b9d945a-cdb7-4d6c-8f24-c2811acfefaf" containerID="d60c4879d39e2f5fdb8436f3411eb29c367fe688bb667a447334cb7aed8f7a2d" exitCode=0
Jul 28 22:16:09.028540 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:09.028424    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-857f577989-92rr5"
Jul 28 22:16:09.028540 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:09.028468    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-857f577989-92rr5" event={"ID":"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf","Type":"ContainerDied","Data":"d60c4879d39e2f5fdb8436f3411eb29c367fe688bb667a447334cb7aed8f7a2d"}
Jul 28 22:16:09.028540 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:09.028507    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-857f577989-92rr5" event={"ID":"9b9d945a-cdb7-4d6c-8f24-c2811acfefaf","Type":"ContainerDied","Data":"1846e0c62d0bcc51090c9ae60b49656b2c9373ab3d7630dc19357d9b6f864615"}
Jul 28 22:16:09.028540 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:09.028525    2583 scope.go:117] "RemoveContainer" containerID="d60c4879d39e2f5fdb8436f3411eb29c367fe688bb667a447334cb7aed8f7a2d"
Jul 28 22:16:09.045419 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:09.045391    2583 scope.go:117] "RemoveContainer" containerID="d60c4879d39e2f5fdb8436f3411eb29c367fe688bb667a447334cb7aed8f7a2d"
Jul 28 22:16:09.045690 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:16:09.045664    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d60c4879d39e2f5fdb8436f3411eb29c367fe688bb667a447334cb7aed8f7a2d\": container with ID starting with d60c4879d39e2f5fdb8436f3411eb29c367fe688bb667a447334cb7aed8f7a2d not found: ID does not exist" containerID="d60c4879d39e2f5fdb8436f3411eb29c367fe688bb667a447334cb7aed8f7a2d"
Jul 28 22:16:09.045745 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:09.045704    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d60c4879d39e2f5fdb8436f3411eb29c367fe688bb667a447334cb7aed8f7a2d"} err="failed to get container status \"d60c4879d39e2f5fdb8436f3411eb29c367fe688bb667a447334cb7aed8f7a2d\": rpc error: code = NotFound desc = could not find container \"d60c4879d39e2f5fdb8436f3411eb29c367fe688bb667a447334cb7aed8f7a2d\": container with ID starting with d60c4879d39e2f5fdb8436f3411eb29c367fe688bb667a447334cb7aed8f7a2d not found: ID does not exist"
Jul 28 22:16:09.065573 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:09.065547    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-857f577989-92rr5"]
Jul 28 22:16:09.069365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:09.069341    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-857f577989-92rr5"]
Jul 28 22:16:09.125291 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:09.125253    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:16:10.194756 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:10.194719    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:16:10.294290 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:10.294260    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:16:10.431812 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:16:10.431775    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b9d945a-cdb7-4d6c-8f24-c2811acfefaf" path="/var/lib/kubelet/pods/9b9d945a-cdb7-4d6c-8f24-c2811acfefaf/volumes"
Jul 28 22:18:15.641707 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:18:15.641675    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:18:16.747170 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:18:16.747138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:19:02.019306 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:19:02.019262    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:19:02.118613 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:19:02.118577    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:19:02.230358 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:19:02.230320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:19:08.249435 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:19:08.249343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:19:08.356197 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:19:08.356162    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:19:08.451118 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:19:08.451085    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:19:08.589704 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:19:08.589606    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:19:14.208638 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:19:14.208592    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:19:14.309354 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:19:14.309323    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:19:16.407083 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:19:16.407049    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:19:16.407901 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:19:16.407883    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:19:53.281573 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:19:53.281536    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:20:26.731964 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:20:26.731924    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:20:26.833764 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:20:26.833724    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:21:59.194763 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:21:59.194723    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:22:35.074113 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:22:35.074037    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:22:35.125729 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:22:35.125689    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:23:32.846192 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:23:32.846160    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:23:32.944136 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:23:32.944105    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:24:16.442223 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:24:16.442195    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:24:16.444698 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:24:16.444681    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:27:06.565788 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:27:06.565747    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:27:34.632169 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:27:34.632132    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:27:45.612032 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:27:45.611998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:28:06.096226 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:28:06.096188    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:28:06.715988 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:28:06.715951    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:28:08.550125 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:28:08.550025    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:28:43.346029 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:28:43.345967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:28:55.373411 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:28:55.373374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:29:16.476035 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:29:16.476010    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:29:16.481149 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:29:16.481130    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:30:00.141419 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:00.141384    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754630-x96gm"]
Jul 28 22:30:00.143845 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:00.141740    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b9d945a-cdb7-4d6c-8f24-c2811acfefaf" containerName="authorino"
Jul 28 22:30:00.143845 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:00.141750    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b9d945a-cdb7-4d6c-8f24-c2811acfefaf" containerName="authorino"
Jul 28 22:30:00.143845 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:00.141760    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4277813-1e72-4863-8eda-6e5fd847b40d" containerName="cleanup"
Jul 28 22:30:00.143845 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:00.141765    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4277813-1e72-4863-8eda-6e5fd847b40d" containerName="cleanup"
Jul 28 22:30:00.143845 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:00.141835    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b9d945a-cdb7-4d6c-8f24-c2811acfefaf" containerName="authorino"
Jul 28 22:30:00.143845 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:00.141846    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4277813-1e72-4863-8eda-6e5fd847b40d" containerName="cleanup"
Jul 28 22:30:00.144779 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:00.144764    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754630-x96gm"
Jul 28 22:30:00.146695 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:00.146675    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:30:00.154157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:00.154141    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754630-x96gm"
Jul 28 22:30:00.160126 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:00.160105    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754630-x96gm"]
Jul 28 22:30:00.290838 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:00.290815    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754630-x96gm"]
Jul 28 22:30:00.308686 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:00.308660    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754630-x96gm" event={"ID":"8e0b0203-f96e-4223-b68c-ae5efa7eb9c2","Type":"ContainerStarted","Data":"c50493ca479a9e548118f1aa6d982dc5b2891ce25068d2b514ef8f9eb99011eb"}
Jul 28 22:30:01.314962 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:01.314926    2583 generic.go:356] "Generic (PLEG): container finished" podID="8e0b0203-f96e-4223-b68c-ae5efa7eb9c2" containerID="b5c55825b5d501c6bd0475a7c218201dc50a5062c8eed6194cbaa2ef7f3717d3" exitCode=0
Jul 28 22:30:01.315313 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:01.314972    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754630-x96gm" event={"ID":"8e0b0203-f96e-4223-b68c-ae5efa7eb9c2","Type":"ContainerDied","Data":"b5c55825b5d501c6bd0475a7c218201dc50a5062c8eed6194cbaa2ef7f3717d3"}
Jul 28 22:30:02.452202 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:02.452176    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754630-x96gm"
Jul 28 22:30:03.326563 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:03.326539    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754630-x96gm"
Jul 28 22:30:03.326738 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:03.326528    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754630-x96gm" event={"ID":"8e0b0203-f96e-4223-b68c-ae5efa7eb9c2","Type":"ContainerDied","Data":"c50493ca479a9e548118f1aa6d982dc5b2891ce25068d2b514ef8f9eb99011eb"}
Jul 28 22:30:03.326738 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:30:03.326690    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c50493ca479a9e548118f1aa6d982dc5b2891ce25068d2b514ef8f9eb99011eb"
Jul 28 22:31:29.642599 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:31:29.642511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:31:51.367609 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:31:51.367572    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:31:54.434525 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:31:54.434494    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:31:54.540486 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:31:54.540454    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:31:54.927462 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:31:54.927428    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:00.666370 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:00.666328    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:00.789064 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:00.789032    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:00.900459 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:00.900415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:01.197157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:01.197122    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:01.392633 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:01.392597    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:03.314203 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:03.314170    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:03.415789 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:03.415754    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:03.554525 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:03.554490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:03.652850 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:03.652816    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:04.026532 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:04.026448    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:04.128397 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:04.128361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:19.993816 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:19.993772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:28.892688 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:28.892648    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv"]
Jul 28 22:32:28.893164 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:28.893021    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e0b0203-f96e-4223-b68c-ae5efa7eb9c2" containerName="cleanup"
Jul 28 22:32:28.893164 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:28.893033    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e0b0203-f96e-4223-b68c-ae5efa7eb9c2" containerName="cleanup"
Jul 28 22:32:28.893164 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:28.893089    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e0b0203-f96e-4223-b68c-ae5efa7eb9c2" containerName="cleanup"
Jul 28 22:32:28.896136 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:28.896121    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv"
Jul 28 22:32:28.898077 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:28.898057    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:32:28.903243 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:28.903218    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv"]
Jul 28 22:32:29.016360 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:29.016330    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd4c318-e91a-4fe2-bcf9-f0af6a12123f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv\" (UID: \"abd4c318-e91a-4fe2-bcf9-f0af6a12123f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv"
Jul 28 22:32:29.117528 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:29.117492    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd4c318-e91a-4fe2-bcf9-f0af6a12123f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv\" (UID: \"abd4c318-e91a-4fe2-bcf9-f0af6a12123f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv"
Jul 28 22:32:29.118156 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:29.118132    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd4c318-e91a-4fe2-bcf9-f0af6a12123f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv\" (UID: \"abd4c318-e91a-4fe2-bcf9-f0af6a12123f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv"
Jul 28 22:32:29.208034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:29.207955    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv"
Jul 28 22:32:29.338063 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:29.338038    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv"]
Jul 28 22:32:29.339717 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:32:29.339689    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabd4c318_e91a_4fe2_bcf9_f0af6a12123f.slice/crio-0fe3f2006704864a3ce421ec6911f8e99f7fc4562023cc14221220488120f11a WatchSource:0}: Error finding container 0fe3f2006704864a3ce421ec6911f8e99f7fc4562023cc14221220488120f11a: Status 404 returned error can't find the container with id 0fe3f2006704864a3ce421ec6911f8e99f7fc4562023cc14221220488120f11a
Jul 28 22:32:30.091384 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:30.091352    2583 generic.go:356] "Generic (PLEG): container finished" podID="abd4c318-e91a-4fe2-bcf9-f0af6a12123f" containerID="5f1be7895ee99f053589efc549379a7a8e21bd9f3c0c0018278984f480a63eee" exitCode=0
Jul 28 22:32:30.091830 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:30.091440    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv" event={"ID":"abd4c318-e91a-4fe2-bcf9-f0af6a12123f","Type":"ContainerDied","Data":"5f1be7895ee99f053589efc549379a7a8e21bd9f3c0c0018278984f480a63eee"}
Jul 28 22:32:30.091830 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:30.091488    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv" event={"ID":"abd4c318-e91a-4fe2-bcf9-f0af6a12123f","Type":"ContainerStarted","Data":"0fe3f2006704864a3ce421ec6911f8e99f7fc4562023cc14221220488120f11a"}
Jul 28 22:32:31.222157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:31.222135    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv"
Jul 28 22:32:31.334922 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:31.334896    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd4c318-e91a-4fe2-bcf9-f0af6a12123f-maas-api-service-ca\") pod \"abd4c318-e91a-4fe2-bcf9-f0af6a12123f\" (UID: \"abd4c318-e91a-4fe2-bcf9-f0af6a12123f\") "
Jul 28 22:32:31.335252 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:31.335229    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abd4c318-e91a-4fe2-bcf9-f0af6a12123f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "abd4c318-e91a-4fe2-bcf9-f0af6a12123f" (UID: "abd4c318-e91a-4fe2-bcf9-f0af6a12123f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:32:31.436285 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:31.436253    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd4c318-e91a-4fe2-bcf9-f0af6a12123f-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:32:32.102861 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:32.102825    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv" event={"ID":"abd4c318-e91a-4fe2-bcf9-f0af6a12123f","Type":"ContainerDied","Data":"0fe3f2006704864a3ce421ec6911f8e99f7fc4562023cc14221220488120f11a"}
Jul 28 22:32:32.103041 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:32.102879    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0fe3f2006704864a3ce421ec6911f8e99f7fc4562023cc14221220488120f11a"
Jul 28 22:32:32.103041 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:32.102837    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv"
Jul 28 22:32:33.999539 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:33.999484    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:34.099304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:34.099266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:34.156164 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:34.156134    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv"]
Jul 28 22:32:34.159527 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:34.159479    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:34.159655 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:34.159579    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-09e3e515-2kdgv"]
Jul 28 22:32:34.297332 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:34.297256    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:34.431090 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:34.431056    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abd4c318-e91a-4fe2-bcf9-f0af6a12123f" path="/var/lib/kubelet/pods/abd4c318-e91a-4fe2-bcf9-f0af6a12123f/volumes"
Jul 28 22:32:34.527663 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:34.527629    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:34.770691 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:34.770657    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:34.876706 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:34.876671    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:34.980385 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:34.980355    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:35.290565 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:35.290534    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:35.886191 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:35.886151    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:37.620497 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.620458    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:37.809741 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.809704    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"]
Jul 28 22:32:37.810240 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.810220    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abd4c318-e91a-4fe2-bcf9-f0af6a12123f" containerName="revoke-keys"
Jul 28 22:32:37.810313 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.810243    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="abd4c318-e91a-4fe2-bcf9-f0af6a12123f" containerName="revoke-keys"
Jul 28 22:32:37.810372 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.810349    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="abd4c318-e91a-4fe2-bcf9-f0af6a12123f" containerName="revoke-keys"
Jul 28 22:32:37.814173 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.814152    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:32:37.816494 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.816472    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tlhn2\""
Jul 28 22:32:37.816626 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.816498    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-6a50116a-serving-cert\""
Jul 28 22:32:37.826122 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.826085    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"]
Jul 28 22:32:37.842077 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.842047    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:37.896208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.896109    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0189fc00-d167-44c0-9826-042f0d21cf6a-maas-api-tls\") pod \"maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg\" (UID: \"0189fc00-d167-44c0-9826-042f0d21cf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:32:37.896208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.896155    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0189fc00-d167-44c0-9826-042f0d21cf6a-openshift-service-ca\") pod \"maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg\" (UID: \"0189fc00-d167-44c0-9826-042f0d21cf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:32:37.896391 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.896212    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n984d\" (UniqueName: \"kubernetes.io/projected/0189fc00-d167-44c0-9826-042f0d21cf6a-kube-api-access-n984d\") pod \"maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg\" (UID: \"0189fc00-d167-44c0-9826-042f0d21cf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:32:37.997340 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.997304    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0189fc00-d167-44c0-9826-042f0d21cf6a-maas-api-tls\") pod \"maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg\" (UID: \"0189fc00-d167-44c0-9826-042f0d21cf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:32:37.997340 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.997345    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0189fc00-d167-44c0-9826-042f0d21cf6a-openshift-service-ca\") pod \"maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg\" (UID: \"0189fc00-d167-44c0-9826-042f0d21cf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:32:37.997587 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.997371    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n984d\" (UniqueName: \"kubernetes.io/projected/0189fc00-d167-44c0-9826-042f0d21cf6a-kube-api-access-n984d\") pod \"maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg\" (UID: \"0189fc00-d167-44c0-9826-042f0d21cf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:32:37.997587 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:32:37.997458    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-serving-cert: secret "maas-api-e2e-migrate-6a50116a-serving-cert" not found
Jul 28 22:32:37.997587 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:32:37.997524    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0189fc00-d167-44c0-9826-042f0d21cf6a-maas-api-tls podName:0189fc00-d167-44c0-9826-042f0d21cf6a nodeName:}" failed. No retries permitted until 2026-07-28 22:32:38.497508475 +0000 UTC m=+1702.505374011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0189fc00-d167-44c0-9826-042f0d21cf6a-maas-api-tls") pod "maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg" (UID: "0189fc00-d167-44c0-9826-042f0d21cf6a") : secret "maas-api-e2e-migrate-6a50116a-serving-cert" not found
Jul 28 22:32:37.998156 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:37.998134    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0189fc00-d167-44c0-9826-042f0d21cf6a-openshift-service-ca\") pod \"maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg\" (UID: \"0189fc00-d167-44c0-9826-042f0d21cf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:32:38.005879 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:38.005846    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n984d\" (UniqueName: \"kubernetes.io/projected/0189fc00-d167-44c0-9826-042f0d21cf6a-kube-api-access-n984d\") pod \"maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg\" (UID: \"0189fc00-d167-44c0-9826-042f0d21cf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:32:38.205990 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:38.205907    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:38.408858 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:38.408765    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:38.502830 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:38.502711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0189fc00-d167-44c0-9826-042f0d21cf6a-maas-api-tls\") pod \"maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg\" (UID: \"0189fc00-d167-44c0-9826-042f0d21cf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:32:38.505265 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:38.505240    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0189fc00-d167-44c0-9826-042f0d21cf6a-maas-api-tls\") pod \"maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg\" (UID: \"0189fc00-d167-44c0-9826-042f0d21cf6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:32:38.511431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:38.511407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:32:38.726971 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:38.726935    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:32:38.858997 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:38.858971    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"]
Jul 28 22:32:38.861226 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:32:38.861195    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0189fc00_d167_44c0_9826_042f0d21cf6a.slice/crio-58535b72537bbd1474d0ecfcf178203227fe613c7fdc4614404e76f9d7bac9c5 WatchSource:0}: Error finding container 58535b72537bbd1474d0ecfcf178203227fe613c7fdc4614404e76f9d7bac9c5: Status 404 returned error can't find the container with id 58535b72537bbd1474d0ecfcf178203227fe613c7fdc4614404e76f9d7bac9c5
Jul 28 22:32:38.862635 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:38.862610    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:32:39.140568 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:39.140533    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg" event={"ID":"0189fc00-d167-44c0-9826-042f0d21cf6a","Type":"ContainerStarted","Data":"58535b72537bbd1474d0ecfcf178203227fe613c7fdc4614404e76f9d7bac9c5"}
Jul 28 22:32:42.158608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:42.158567    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg" event={"ID":"0189fc00-d167-44c0-9826-042f0d21cf6a","Type":"ContainerStarted","Data":"6ca839a768d48d4b4e7c108d1547b783d897bb8a4889c1b60cc9c46e2ed41584"}
Jul 28 22:32:42.159056 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:42.158720    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:32:42.176033 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:42.175978    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg" podStartSLOduration=2.843647549 podStartE2EDuration="5.175959722s" totalImagesPullingTime="2.332312173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:32:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:32:42.173340307 +0000 UTC m=+1706.181205865" watchObservedRunningTime="2026-07-28 22:32:42.175959722 +0000 UTC m=+1706.183825281"
Jul 28 22:32:48.169763 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:32:48.169733    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:33:02.626848 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:02.626812    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm"]
Jul 28 22:33:02.630342 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:02.630327    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm"
Jul 28 22:33:02.632477 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:02.632458    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:33:02.637086 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:02.637052    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm"]
Jul 28 22:33:02.700889 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:02.700853    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa33d63-4609-48e9-878e-3adaf51dfa6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm\" (UID: \"8aa33d63-4609-48e9-878e-3adaf51dfa6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm"
Jul 28 22:33:02.801298 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:02.801258    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa33d63-4609-48e9-878e-3adaf51dfa6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm\" (UID: \"8aa33d63-4609-48e9-878e-3adaf51dfa6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm"
Jul 28 22:33:02.801997 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:02.801976    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa33d63-4609-48e9-878e-3adaf51dfa6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm\" (UID: \"8aa33d63-4609-48e9-878e-3adaf51dfa6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm"
Jul 28 22:33:02.942136 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:02.942051    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm"
Jul 28 22:33:03.092627 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:03.092602    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm"]
Jul 28 22:33:03.094470 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:33:03.094429    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aa33d63_4609_48e9_878e_3adaf51dfa6a.slice/crio-ed46063456c5bce3da3eb2a8ea597d46dcfa95db4e0ccc198fd62720074cdcfc WatchSource:0}: Error finding container ed46063456c5bce3da3eb2a8ea597d46dcfa95db4e0ccc198fd62720074cdcfc: Status 404 returned error can't find the container with id ed46063456c5bce3da3eb2a8ea597d46dcfa95db4e0ccc198fd62720074cdcfc
Jul 28 22:33:03.271159 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:03.271068    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm" event={"ID":"8aa33d63-4609-48e9-878e-3adaf51dfa6a","Type":"ContainerStarted","Data":"ed46063456c5bce3da3eb2a8ea597d46dcfa95db4e0ccc198fd62720074cdcfc"}
Jul 28 22:33:04.278040 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:04.278001    2583 generic.go:356] "Generic (PLEG): container finished" podID="8aa33d63-4609-48e9-878e-3adaf51dfa6a" containerID="c43e50ada620b7c23905f7bd226c260a96b25645c288992a32a8858629ee806d" exitCode=0
Jul 28 22:33:04.278422 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:04.278089    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm" event={"ID":"8aa33d63-4609-48e9-878e-3adaf51dfa6a","Type":"ContainerDied","Data":"c43e50ada620b7c23905f7bd226c260a96b25645c288992a32a8858629ee806d"}
Jul 28 22:33:05.419341 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:05.419309    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm"
Jul 28 22:33:05.419738 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:05.419674    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa33d63-4609-48e9-878e-3adaf51dfa6a-maas-api-service-ca\") pod \"8aa33d63-4609-48e9-878e-3adaf51dfa6a\" (UID: \"8aa33d63-4609-48e9-878e-3adaf51dfa6a\") "
Jul 28 22:33:05.420000 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:05.419977    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8aa33d63-4609-48e9-878e-3adaf51dfa6a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8aa33d63-4609-48e9-878e-3adaf51dfa6a" (UID: "8aa33d63-4609-48e9-878e-3adaf51dfa6a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:33:05.520373 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:05.520343    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa33d63-4609-48e9-878e-3adaf51dfa6a-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:33:06.290186 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:06.290161    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm"
Jul 28 22:33:06.290350 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:06.290188    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm" event={"ID":"8aa33d63-4609-48e9-878e-3adaf51dfa6a","Type":"ContainerDied","Data":"ed46063456c5bce3da3eb2a8ea597d46dcfa95db4e0ccc198fd62720074cdcfc"}
Jul 28 22:33:06.290350 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:06.290227    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed46063456c5bce3da3eb2a8ea597d46dcfa95db4e0ccc198fd62720074cdcfc"
Jul 28 22:33:09.011655 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.011616    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:09.113356 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.113326    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:09.123927 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.123897    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"]
Jul 28 22:33:09.124124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.124099    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg" podUID="0189fc00-d167-44c0-9826-042f0d21cf6a" containerName="maas-api" containerID="cri-o://6ca839a768d48d4b4e7c108d1547b783d897bb8a4889c1b60cc9c46e2ed41584" gracePeriod=30
Jul 28 22:33:09.182958 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.182927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:09.246517 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.246491    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:09.310677 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.310646    2583 generic.go:356] "Generic (PLEG): container finished" podID="0189fc00-d167-44c0-9826-042f0d21cf6a" containerID="6ca839a768d48d4b4e7c108d1547b783d897bb8a4889c1b60cc9c46e2ed41584" exitCode=0
Jul 28 22:33:09.310876 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.310713    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg" event={"ID":"0189fc00-d167-44c0-9826-042f0d21cf6a","Type":"ContainerDied","Data":"6ca839a768d48d4b4e7c108d1547b783d897bb8a4889c1b60cc9c46e2ed41584"}
Jul 28 22:33:09.328685 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.328652    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm"]
Jul 28 22:33:09.330313 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.330288    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:09.330983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.330964    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6a50116a-d5pbm"]
Jul 28 22:33:09.377126 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.377104    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:33:09.449654 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.449624    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0189fc00-d167-44c0-9826-042f0d21cf6a-openshift-service-ca\") pod \"0189fc00-d167-44c0-9826-042f0d21cf6a\" (UID: \"0189fc00-d167-44c0-9826-042f0d21cf6a\") "
Jul 28 22:33:09.449848 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.449693    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n984d\" (UniqueName: \"kubernetes.io/projected/0189fc00-d167-44c0-9826-042f0d21cf6a-kube-api-access-n984d\") pod \"0189fc00-d167-44c0-9826-042f0d21cf6a\" (UID: \"0189fc00-d167-44c0-9826-042f0d21cf6a\") "
Jul 28 22:33:09.449848 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.449734    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0189fc00-d167-44c0-9826-042f0d21cf6a-maas-api-tls\") pod \"0189fc00-d167-44c0-9826-042f0d21cf6a\" (UID: \"0189fc00-d167-44c0-9826-042f0d21cf6a\") "
Jul 28 22:33:09.450088 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.450057    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0189fc00-d167-44c0-9826-042f0d21cf6a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0189fc00-d167-44c0-9826-042f0d21cf6a" (UID: "0189fc00-d167-44c0-9826-042f0d21cf6a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:33:09.452240 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.452205    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0189fc00-d167-44c0-9826-042f0d21cf6a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0189fc00-d167-44c0-9826-042f0d21cf6a" (UID: "0189fc00-d167-44c0-9826-042f0d21cf6a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:33:09.452240 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.452216    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0189fc00-d167-44c0-9826-042f0d21cf6a-kube-api-access-n984d" (OuterVolumeSpecName: "kube-api-access-n984d") pod "0189fc00-d167-44c0-9826-042f0d21cf6a" (UID: "0189fc00-d167-44c0-9826-042f0d21cf6a"). InnerVolumeSpecName "kube-api-access-n984d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:33:09.482030 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.481997    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:09.550928 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.550842    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n984d\" (UniqueName: \"kubernetes.io/projected/0189fc00-d167-44c0-9826-042f0d21cf6a-kube-api-access-n984d\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:33:09.550928 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.550871    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0189fc00-d167-44c0-9826-042f0d21cf6a-maas-api-tls\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:33:09.550928 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.550887    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0189fc00-d167-44c0-9826-042f0d21cf6a-openshift-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:33:09.718860 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.718788    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:09.943124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:09.943069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:10.269276 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:10.269176    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:10.317575 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:10.317543    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg" event={"ID":"0189fc00-d167-44c0-9826-042f0d21cf6a","Type":"ContainerDied","Data":"58535b72537bbd1474d0ecfcf178203227fe613c7fdc4614404e76f9d7bac9c5"}
Jul 28 22:33:10.317575 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:10.317569    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"
Jul 28 22:33:10.317761 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:10.317626    2583 scope.go:117] "RemoveContainer" containerID="6ca839a768d48d4b4e7c108d1547b783d897bb8a4889c1b60cc9c46e2ed41584"
Jul 28 22:33:10.342160 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:10.342134    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"]
Jul 28 22:33:10.347356 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:10.347325    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6a50116a-57bcff8486-rddwg"]
Jul 28 22:33:10.431593 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:10.431561    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0189fc00-d167-44c0-9826-042f0d21cf6a" path="/var/lib/kubelet/pods/0189fc00-d167-44c0-9826-042f0d21cf6a/volumes"
Jul 28 22:33:10.431926 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:10.431912    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8aa33d63-4609-48e9-878e-3adaf51dfa6a" path="/var/lib/kubelet/pods/8aa33d63-4609-48e9-878e-3adaf51dfa6a/volumes"
Jul 28 22:33:10.626373 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:10.626313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:12.696298 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.696265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:12.792881 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.792850    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:12.798642 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.798619    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"]
Jul 28 22:33:12.799008 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.798988    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aa33d63-4609-48e9-878e-3adaf51dfa6a" containerName="revoke-keys"
Jul 28 22:33:12.799008 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.799003    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aa33d63-4609-48e9-878e-3adaf51dfa6a" containerName="revoke-keys"
Jul 28 22:33:12.799169 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.799014    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0189fc00-d167-44c0-9826-042f0d21cf6a" containerName="maas-api"
Jul 28 22:33:12.799169 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.799020    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0189fc00-d167-44c0-9826-042f0d21cf6a" containerName="maas-api"
Jul 28 22:33:12.799169 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.799075    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0189fc00-d167-44c0-9826-042f0d21cf6a" containerName="maas-api"
Jul 28 22:33:12.799169 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.799085    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aa33d63-4609-48e9-878e-3adaf51dfa6a" containerName="revoke-keys"
Jul 28 22:33:12.803282 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.803262    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:12.805327 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.805310    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tlhn2\""
Jul 28 22:33:12.805418 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.805351    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e1a8249b-serving-cert\""
Jul 28 22:33:12.812568 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.812546    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"]
Jul 28 22:33:12.856252 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.856229    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:12.872422 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.872393    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab97a4a5-811e-4dd4-b780-a41d87c123a2-openshift-service-ca\") pod \"maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8\" (UID: \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:12.872534 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.872427    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqm4g\" (UniqueName: \"kubernetes.io/projected/ab97a4a5-811e-4dd4-b780-a41d87c123a2-kube-api-access-dqm4g\") pod \"maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8\" (UID: \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:12.872534 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.872476    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab97a4a5-811e-4dd4-b780-a41d87c123a2-maas-api-tls\") pod \"maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8\" (UID: \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:12.973425 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.973357    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab97a4a5-811e-4dd4-b780-a41d87c123a2-maas-api-tls\") pod \"maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8\" (UID: \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:12.973425 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.973415    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab97a4a5-811e-4dd4-b780-a41d87c123a2-openshift-service-ca\") pod \"maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8\" (UID: \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:12.973567 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.973437    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqm4g\" (UniqueName: \"kubernetes.io/projected/ab97a4a5-811e-4dd4-b780-a41d87c123a2-kube-api-access-dqm4g\") pod \"maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8\" (UID: \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:12.974117 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.974093    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab97a4a5-811e-4dd4-b780-a41d87c123a2-openshift-service-ca\") pod \"maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8\" (UID: \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:12.975826 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.975774    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab97a4a5-811e-4dd4-b780-a41d87c123a2-maas-api-tls\") pod \"maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8\" (UID: \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:12.983104 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:12.983085    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqm4g\" (UniqueName: \"kubernetes.io/projected/ab97a4a5-811e-4dd4-b780-a41d87c123a2-kube-api-access-dqm4g\") pod \"maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8\" (UID: \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:13.003692 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:13.003668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:13.116474 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:13.116445    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:13.252687 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:13.252659    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"]
Jul 28 22:33:13.254962 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:33:13.254936    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab97a4a5_811e_4dd4_b780_a41d87c123a2.slice/crio-815a0086b8d58cc6388037873cc3c1fdf5ef1b08180fa5d0322935254068b349 WatchSource:0}: Error finding container 815a0086b8d58cc6388037873cc3c1fdf5ef1b08180fa5d0322935254068b349: Status 404 returned error can't find the container with id 815a0086b8d58cc6388037873cc3c1fdf5ef1b08180fa5d0322935254068b349
Jul 28 22:33:13.334383 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:13.334346    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8" event={"ID":"ab97a4a5-811e-4dd4-b780-a41d87c123a2","Type":"ContainerStarted","Data":"815a0086b8d58cc6388037873cc3c1fdf5ef1b08180fa5d0322935254068b349"}
Jul 28 22:33:13.464085 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:13.464048    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:13.562626 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:13.562525    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:14.342244 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:14.342210    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8" event={"ID":"ab97a4a5-811e-4dd4-b780-a41d87c123a2","Type":"ContainerStarted","Data":"a735fcfed1b76ecae7ddebbcd4150ffd9b2bb123a7f2981aafa09762d2f4e830"}
Jul 28 22:33:14.342633 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:14.342312    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:14.358573 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:14.358530    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8" podStartSLOduration=1.974585975 podStartE2EDuration="2.35851797s" totalImagesPullingTime="383.931995ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:33:14.356877285 +0000 UTC m=+1738.364743034" watchObservedRunningTime="2026-07-28 22:33:14.35851797 +0000 UTC m=+1738.366383555"
Jul 28 22:33:20.353858 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:20.353825    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:38.497580 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:38.497542    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4"]
Jul 28 22:33:38.501105 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:38.501087    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4"
Jul 28 22:33:38.503002 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:38.502980    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:33:38.507159 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:38.507135    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4"]
Jul 28 22:33:38.590240 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:38.590188    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/475390e6-5799-4cee-b2e0-a384a1259b6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4\" (UID: \"475390e6-5799-4cee-b2e0-a384a1259b6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4"
Jul 28 22:33:38.690955 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:38.690925    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/475390e6-5799-4cee-b2e0-a384a1259b6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4\" (UID: \"475390e6-5799-4cee-b2e0-a384a1259b6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4"
Jul 28 22:33:38.691526 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:38.691504    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/475390e6-5799-4cee-b2e0-a384a1259b6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4\" (UID: \"475390e6-5799-4cee-b2e0-a384a1259b6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4"
Jul 28 22:33:38.813347 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:38.813268    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4"
Jul 28 22:33:38.946672 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:38.946647    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4"]
Jul 28 22:33:38.948591 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:33:38.948546    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod475390e6_5799_4cee_b2e0_a384a1259b6e.slice/crio-dc6619b01120db9f00bd354079383c73f7d2df4eb2258b1b12b29b259bd97bff WatchSource:0}: Error finding container dc6619b01120db9f00bd354079383c73f7d2df4eb2258b1b12b29b259bd97bff: Status 404 returned error can't find the container with id dc6619b01120db9f00bd354079383c73f7d2df4eb2258b1b12b29b259bd97bff
Jul 28 22:33:39.475853 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:39.475727    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4" event={"ID":"475390e6-5799-4cee-b2e0-a384a1259b6e","Type":"ContainerStarted","Data":"1b1b55927890ad8f696547b48a4a1bb123c4ad518c45dadbcf11689542fedbcd"}
Jul 28 22:33:39.475853 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:39.475786    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4" event={"ID":"475390e6-5799-4cee-b2e0-a384a1259b6e","Type":"ContainerStarted","Data":"dc6619b01120db9f00bd354079383c73f7d2df4eb2258b1b12b29b259bd97bff"}
Jul 28 22:33:39.490150 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:39.490101    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4" podStartSLOduration=1.490085311 podStartE2EDuration="1.490085311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:33:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:33:39.487933439 +0000 UTC m=+1763.495798993" watchObservedRunningTime="2026-07-28 22:33:39.490085311 +0000 UTC m=+1763.497950866"
Jul 28 22:33:40.482470 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:40.482431    2583 generic.go:356] "Generic (PLEG): container finished" podID="475390e6-5799-4cee-b2e0-a384a1259b6e" containerID="1b1b55927890ad8f696547b48a4a1bb123c4ad518c45dadbcf11689542fedbcd" exitCode=0
Jul 28 22:33:40.482885 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:40.482514    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4" event={"ID":"475390e6-5799-4cee-b2e0-a384a1259b6e","Type":"ContainerDied","Data":"1b1b55927890ad8f696547b48a4a1bb123c4ad518c45dadbcf11689542fedbcd"}
Jul 28 22:33:41.615443 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:41.615419    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4"
Jul 28 22:33:41.718771 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:41.718723    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/475390e6-5799-4cee-b2e0-a384a1259b6e-maas-api-service-ca\") pod \"475390e6-5799-4cee-b2e0-a384a1259b6e\" (UID: \"475390e6-5799-4cee-b2e0-a384a1259b6e\") "
Jul 28 22:33:41.719185 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:41.719154    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/475390e6-5799-4cee-b2e0-a384a1259b6e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "475390e6-5799-4cee-b2e0-a384a1259b6e" (UID: "475390e6-5799-4cee-b2e0-a384a1259b6e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:33:41.819751 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:41.819662    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/475390e6-5799-4cee-b2e0-a384a1259b6e-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:33:42.494131 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:42.494100    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4"
Jul 28 22:33:42.494131 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:42.494111    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4" event={"ID":"475390e6-5799-4cee-b2e0-a384a1259b6e","Type":"ContainerDied","Data":"dc6619b01120db9f00bd354079383c73f7d2df4eb2258b1b12b29b259bd97bff"}
Jul 28 22:33:42.494339 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:42.494165    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc6619b01120db9f00bd354079383c73f7d2df4eb2258b1b12b29b259bd97bff"
Jul 28 22:33:43.562423 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.562389    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"]
Jul 28 22:33:43.562915 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.562591    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8" podUID="ab97a4a5-811e-4dd4-b780-a41d87c123a2" containerName="maas-api" containerID="cri-o://a735fcfed1b76ecae7ddebbcd4150ffd9b2bb123a7f2981aafa09762d2f4e830" gracePeriod=30
Jul 28 22:33:43.594834 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.594750    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:43.695396 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.695345    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:43.757844 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.757809    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4"]
Jul 28 22:33:43.761874 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.761844    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:43.763326 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.763307    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e1a8249b-pgjf4"]
Jul 28 22:33:43.814712 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.814649    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:43.937370 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.937338    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab97a4a5-811e-4dd4-b780-a41d87c123a2-openshift-service-ca\") pod \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\" (UID: \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\") "
Jul 28 22:33:43.937566 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.937392    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqm4g\" (UniqueName: \"kubernetes.io/projected/ab97a4a5-811e-4dd4-b780-a41d87c123a2-kube-api-access-dqm4g\") pod \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\" (UID: \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\") "
Jul 28 22:33:43.937566 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.937516    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab97a4a5-811e-4dd4-b780-a41d87c123a2-maas-api-tls\") pod \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\" (UID: \"ab97a4a5-811e-4dd4-b780-a41d87c123a2\") "
Jul 28 22:33:43.937814 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.937772    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab97a4a5-811e-4dd4-b780-a41d87c123a2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ab97a4a5-811e-4dd4-b780-a41d87c123a2" (UID: "ab97a4a5-811e-4dd4-b780-a41d87c123a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:33:43.939515 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.939488    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab97a4a5-811e-4dd4-b780-a41d87c123a2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ab97a4a5-811e-4dd4-b780-a41d87c123a2" (UID: "ab97a4a5-811e-4dd4-b780-a41d87c123a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:33:43.939611 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:43.939535    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab97a4a5-811e-4dd4-b780-a41d87c123a2-kube-api-access-dqm4g" (OuterVolumeSpecName: "kube-api-access-dqm4g") pod "ab97a4a5-811e-4dd4-b780-a41d87c123a2" (UID: "ab97a4a5-811e-4dd4-b780-a41d87c123a2"). InnerVolumeSpecName "kube-api-access-dqm4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:33:44.038980 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.038940    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab97a4a5-811e-4dd4-b780-a41d87c123a2-openshift-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:33:44.038980 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.038974    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqm4g\" (UniqueName: \"kubernetes.io/projected/ab97a4a5-811e-4dd4-b780-a41d87c123a2-kube-api-access-dqm4g\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:33:44.038980 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.038986    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab97a4a5-811e-4dd4-b780-a41d87c123a2-maas-api-tls\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:33:44.431550 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.431515    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="475390e6-5799-4cee-b2e0-a384a1259b6e" path="/var/lib/kubelet/pods/475390e6-5799-4cee-b2e0-a384a1259b6e/volumes"
Jul 28 22:33:44.506087 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.506058    2583 generic.go:356] "Generic (PLEG): container finished" podID="ab97a4a5-811e-4dd4-b780-a41d87c123a2" containerID="a735fcfed1b76ecae7ddebbcd4150ffd9b2bb123a7f2981aafa09762d2f4e830" exitCode=0
Jul 28 22:33:44.506260 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.506105    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"
Jul 28 22:33:44.506260 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.506139    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8" event={"ID":"ab97a4a5-811e-4dd4-b780-a41d87c123a2","Type":"ContainerDied","Data":"a735fcfed1b76ecae7ddebbcd4150ffd9b2bb123a7f2981aafa09762d2f4e830"}
Jul 28 22:33:44.506260 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.506192    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8" event={"ID":"ab97a4a5-811e-4dd4-b780-a41d87c123a2","Type":"ContainerDied","Data":"815a0086b8d58cc6388037873cc3c1fdf5ef1b08180fa5d0322935254068b349"}
Jul 28 22:33:44.506260 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.506213    2583 scope.go:117] "RemoveContainer" containerID="a735fcfed1b76ecae7ddebbcd4150ffd9b2bb123a7f2981aafa09762d2f4e830"
Jul 28 22:33:44.515089 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.515070    2583 scope.go:117] "RemoveContainer" containerID="a735fcfed1b76ecae7ddebbcd4150ffd9b2bb123a7f2981aafa09762d2f4e830"
Jul 28 22:33:44.515316 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:33:44.515299    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a735fcfed1b76ecae7ddebbcd4150ffd9b2bb123a7f2981aafa09762d2f4e830\": container with ID starting with a735fcfed1b76ecae7ddebbcd4150ffd9b2bb123a7f2981aafa09762d2f4e830 not found: ID does not exist" containerID="a735fcfed1b76ecae7ddebbcd4150ffd9b2bb123a7f2981aafa09762d2f4e830"
Jul 28 22:33:44.515382 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.515322    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a735fcfed1b76ecae7ddebbcd4150ffd9b2bb123a7f2981aafa09762d2f4e830"} err="failed to get container status \"a735fcfed1b76ecae7ddebbcd4150ffd9b2bb123a7f2981aafa09762d2f4e830\": rpc error: code = NotFound desc = could not find container \"a735fcfed1b76ecae7ddebbcd4150ffd9b2bb123a7f2981aafa09762d2f4e830\": container with ID starting with a735fcfed1b76ecae7ddebbcd4150ffd9b2bb123a7f2981aafa09762d2f4e830 not found: ID does not exist"
Jul 28 22:33:44.521890 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.521867    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"]
Jul 28 22:33:44.526893 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.526874    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e1a8249b-8599bfcb8d-crjc8"]
Jul 28 22:33:44.902189 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:44.902160    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:45.120666 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:45.120634    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:45.380018 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:45.379988    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:45.684789 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:45.684713    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:46.206120 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:46.206091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:46.432028 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:46.431996    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab97a4a5-811e-4dd4-b780-a41d87c123a2" path="/var/lib/kubelet/pods/ab97a4a5-811e-4dd4-b780-a41d87c123a2/volumes"
Jul 28 22:33:47.964323 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:47.964287    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:48.053325 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.053295    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:48.110762 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.110733    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"]
Jul 28 22:33:48.111129 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.111116    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab97a4a5-811e-4dd4-b780-a41d87c123a2" containerName="maas-api"
Jul 28 22:33:48.111176 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.111131    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab97a4a5-811e-4dd4-b780-a41d87c123a2" containerName="maas-api"
Jul 28 22:33:48.111176 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.111142    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="475390e6-5799-4cee-b2e0-a384a1259b6e" containerName="revoke-keys"
Jul 28 22:33:48.111176 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.111147    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="475390e6-5799-4cee-b2e0-a384a1259b6e" containerName="revoke-keys"
Jul 28 22:33:48.111268 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.111200    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab97a4a5-811e-4dd4-b780-a41d87c123a2" containerName="maas-api"
Jul 28 22:33:48.111268 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.111211    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="475390e6-5799-4cee-b2e0-a384a1259b6e" containerName="revoke-keys"
Jul 28 22:33:48.114124 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.114110    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:33:48.121316 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.121292    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-f8361843-serving-cert\""
Jul 28 22:33:48.121467 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.121300    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tlhn2\""
Jul 28 22:33:48.126321 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.126297    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"]
Jul 28 22:33:48.175612 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.175585    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-maas-api-tls\") pod \"maas-api-e2e-derive-f8361843-55d97ccdb8-sf554\" (UID: \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:33:48.175612 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.175618    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqlvv\" (UniqueName: \"kubernetes.io/projected/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-kube-api-access-dqlvv\") pod \"maas-api-e2e-derive-f8361843-55d97ccdb8-sf554\" (UID: \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:33:48.175849 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.175661    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-openshift-service-ca\") pod \"maas-api-e2e-derive-f8361843-55d97ccdb8-sf554\" (UID: \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:33:48.234418 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.234351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:48.276649 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.276623    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-maas-api-tls\") pod \"maas-api-e2e-derive-f8361843-55d97ccdb8-sf554\" (UID: \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:33:48.276757 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.276652    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqlvv\" (UniqueName: \"kubernetes.io/projected/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-kube-api-access-dqlvv\") pod \"maas-api-e2e-derive-f8361843-55d97ccdb8-sf554\" (UID: \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:33:48.276757 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.276679    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-openshift-service-ca\") pod \"maas-api-e2e-derive-f8361843-55d97ccdb8-sf554\" (UID: \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:33:48.276870 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:33:48.276757    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-serving-cert: secret "maas-api-e2e-derive-f8361843-serving-cert" not found
Jul 28 22:33:48.276870 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:33:48.276836    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-maas-api-tls podName:5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f nodeName:}" failed. No retries permitted until 2026-07-28 22:33:48.776816409 +0000 UTC m=+1772.784681952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-maas-api-tls") pod "maas-api-e2e-derive-f8361843-55d97ccdb8-sf554" (UID: "5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f") : secret "maas-api-e2e-derive-f8361843-serving-cert" not found
Jul 28 22:33:48.277273 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.277256    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-openshift-service-ca\") pod \"maas-api-e2e-derive-f8361843-55d97ccdb8-sf554\" (UID: \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:33:48.287708 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.287691    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqlvv\" (UniqueName: \"kubernetes.io/projected/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-kube-api-access-dqlvv\") pod \"maas-api-e2e-derive-f8361843-55d97ccdb8-sf554\" (UID: \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:33:48.374910 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.374884    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:48.722872 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.722842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:48.780700 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.780668    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-maas-api-tls\") pod \"maas-api-e2e-derive-f8361843-55d97ccdb8-sf554\" (UID: \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:33:48.783060 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.783040    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-maas-api-tls\") pod \"maas-api-e2e-derive-f8361843-55d97ccdb8-sf554\" (UID: \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:33:48.824995 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:48.824961    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:33:49.032600 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:49.032512    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:33:49.170330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:49.170305    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"]
Jul 28 22:33:49.172075 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:33:49.172040    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b3d7f61_8af3_4b40_9ea6_8229ebcd2c0f.slice/crio-a02f448549d8e77b2079f2ccddbe8b4caf61a88270b44a5a96d045861b444692 WatchSource:0}: Error finding container a02f448549d8e77b2079f2ccddbe8b4caf61a88270b44a5a96d045861b444692: Status 404 returned error can't find the container with id a02f448549d8e77b2079f2ccddbe8b4caf61a88270b44a5a96d045861b444692
Jul 28 22:33:49.538592 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:49.538567    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554" event={"ID":"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f","Type":"ContainerStarted","Data":"a02f448549d8e77b2079f2ccddbe8b4caf61a88270b44a5a96d045861b444692"}
Jul 28 22:33:50.544828 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:50.544774    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554" event={"ID":"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f","Type":"ContainerStarted","Data":"fefdbc7f469c9b849939947d1c82ce3aa61c5aaec47e53951c60416ad672d6e4"}
Jul 28 22:33:50.545217 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:50.544931    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:33:50.561205 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:50.561160    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554" podStartSLOduration=2.2492056910000002 podStartE2EDuration="2.561145208s" totalImagesPullingTime="311.939517ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:33:50.559956475 +0000 UTC m=+1774.567822059" watchObservedRunningTime="2026-07-28 22:33:50.561145208 +0000 UTC m=+1774.569010767"
Jul 28 22:33:56.555788 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:33:56.555752    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:34:03.658770 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:03.658728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:12.764857 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:12.764825    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk"]
Jul 28 22:34:12.768641 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:12.768620    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk"
Jul 28 22:34:12.770555 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:12.770528    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:34:12.776179 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:12.776157    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk"]
Jul 28 22:34:12.785888 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:12.785866    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f020a4b0-14b8-440a-ba36-4483960d9a28-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f8361843-dvdhk\" (UID: \"f020a4b0-14b8-440a-ba36-4483960d9a28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk"
Jul 28 22:34:12.886906 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:12.886873    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f020a4b0-14b8-440a-ba36-4483960d9a28-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f8361843-dvdhk\" (UID: \"f020a4b0-14b8-440a-ba36-4483960d9a28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk"
Jul 28 22:34:12.887488 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:12.887466    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f020a4b0-14b8-440a-ba36-4483960d9a28-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f8361843-dvdhk\" (UID: \"f020a4b0-14b8-440a-ba36-4483960d9a28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk"
Jul 28 22:34:13.079952 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:13.079871    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk"
Jul 28 22:34:13.417839 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:13.417812    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk"]
Jul 28 22:34:13.419663 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:34:13.419632    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf020a4b0_14b8_440a_ba36_4483960d9a28.slice/crio-675d33c9a8da448b29a114426e0fed826258d0f29f58cb7fb92ba26bb813ba1a WatchSource:0}: Error finding container 675d33c9a8da448b29a114426e0fed826258d0f29f58cb7fb92ba26bb813ba1a: Status 404 returned error can't find the container with id 675d33c9a8da448b29a114426e0fed826258d0f29f58cb7fb92ba26bb813ba1a
Jul 28 22:34:13.669097 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:13.669031    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk" event={"ID":"f020a4b0-14b8-440a-ba36-4483960d9a28","Type":"ContainerStarted","Data":"675d33c9a8da448b29a114426e0fed826258d0f29f58cb7fb92ba26bb813ba1a"}
Jul 28 22:34:14.675566 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:14.675530    2583 generic.go:356] "Generic (PLEG): container finished" podID="f020a4b0-14b8-440a-ba36-4483960d9a28" containerID="c0ca5c3a0ef542f7be4bbb91db8a6a9d82624d154a37ff1ce871eccdb89fe4fa" exitCode=0
Jul 28 22:34:14.676010 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:14.675617    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk" event={"ID":"f020a4b0-14b8-440a-ba36-4483960d9a28","Type":"ContainerDied","Data":"c0ca5c3a0ef542f7be4bbb91db8a6a9d82624d154a37ff1ce871eccdb89fe4fa"}
Jul 28 22:34:15.808583 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:15.808559    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk"
Jul 28 22:34:15.912380 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:15.912343    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f020a4b0-14b8-440a-ba36-4483960d9a28-maas-api-service-ca\") pod \"f020a4b0-14b8-440a-ba36-4483960d9a28\" (UID: \"f020a4b0-14b8-440a-ba36-4483960d9a28\") "
Jul 28 22:34:15.912693 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:15.912671    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f020a4b0-14b8-440a-ba36-4483960d9a28-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f020a4b0-14b8-440a-ba36-4483960d9a28" (UID: "f020a4b0-14b8-440a-ba36-4483960d9a28"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:34:16.013904 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:16.013771    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f020a4b0-14b8-440a-ba36-4483960d9a28-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:34:16.512432 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:16.512408    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:34:16.518937 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:16.518913    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:34:16.686894 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:16.686865    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk"
Jul 28 22:34:16.687078 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:16.686908    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk" event={"ID":"f020a4b0-14b8-440a-ba36-4483960d9a28","Type":"ContainerDied","Data":"675d33c9a8da448b29a114426e0fed826258d0f29f58cb7fb92ba26bb813ba1a"}
Jul 28 22:34:16.687078 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:16.686982    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="675d33c9a8da448b29a114426e0fed826258d0f29f58cb7fb92ba26bb813ba1a"
Jul 28 22:34:18.620208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.620157    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"]
Jul 28 22:34:18.620616 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.620393    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554" podUID="5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f" containerName="maas-api" containerID="cri-o://fefdbc7f469c9b849939947d1c82ce3aa61c5aaec47e53951c60416ad672d6e4" gracePeriod=30
Jul 28 22:34:18.647602 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.647572    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:18.723318 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.723292    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:18.788178 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.787972    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:18.825430 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.825398    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk"]
Jul 28 22:34:18.828346 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.828322    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f8361843-dvdhk"]
Jul 28 22:34:18.867453 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.867434    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:34:18.906674 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.906607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:18.938397 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.938378    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-maas-api-tls\") pod \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\" (UID: \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\") "
Jul 28 22:34:18.938490 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.938415    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-openshift-service-ca\") pod \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\" (UID: \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\") "
Jul 28 22:34:18.938490 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.938445    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqlvv\" (UniqueName: \"kubernetes.io/projected/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-kube-api-access-dqlvv\") pod \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\" (UID: \"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f\") "
Jul 28 22:34:18.938855 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.938834    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f" (UID: "5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:34:18.941057 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.941039    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f" (UID: "5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:34:18.941141 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:18.941118    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-kube-api-access-dqlvv" (OuterVolumeSpecName: "kube-api-access-dqlvv") pod "5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f" (UID: "5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f"). InnerVolumeSpecName "kube-api-access-dqlvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:34:19.039710 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.039680    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-maas-api-tls\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:34:19.039710 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.039704    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-openshift-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:34:19.039710 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.039715    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqlvv\" (UniqueName: \"kubernetes.io/projected/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f-kube-api-access-dqlvv\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:34:19.084956 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.084929    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:19.226262 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.226191    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:19.695778 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.695745    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:19.706975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.706948    2583 generic.go:356] "Generic (PLEG): container finished" podID="5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f" containerID="fefdbc7f469c9b849939947d1c82ce3aa61c5aaec47e53951c60416ad672d6e4" exitCode=0
Jul 28 22:34:19.707102 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.706995    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"
Jul 28 22:34:19.707102 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.707025    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554" event={"ID":"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f","Type":"ContainerDied","Data":"fefdbc7f469c9b849939947d1c82ce3aa61c5aaec47e53951c60416ad672d6e4"}
Jul 28 22:34:19.707102 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.707084    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554" event={"ID":"5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f","Type":"ContainerDied","Data":"a02f448549d8e77b2079f2ccddbe8b4caf61a88270b44a5a96d045861b444692"}
Jul 28 22:34:19.707228 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.707113    2583 scope.go:117] "RemoveContainer" containerID="fefdbc7f469c9b849939947d1c82ce3aa61c5aaec47e53951c60416ad672d6e4"
Jul 28 22:34:19.716675 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.716659    2583 scope.go:117] "RemoveContainer" containerID="fefdbc7f469c9b849939947d1c82ce3aa61c5aaec47e53951c60416ad672d6e4"
Jul 28 22:34:19.716946 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:34:19.716928    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fefdbc7f469c9b849939947d1c82ce3aa61c5aaec47e53951c60416ad672d6e4\": container with ID starting with fefdbc7f469c9b849939947d1c82ce3aa61c5aaec47e53951c60416ad672d6e4 not found: ID does not exist" containerID="fefdbc7f469c9b849939947d1c82ce3aa61c5aaec47e53951c60416ad672d6e4"
Jul 28 22:34:19.717013 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.716953    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fefdbc7f469c9b849939947d1c82ce3aa61c5aaec47e53951c60416ad672d6e4"} err="failed to get container status \"fefdbc7f469c9b849939947d1c82ce3aa61c5aaec47e53951c60416ad672d6e4\": rpc error: code = NotFound desc = could not find container \"fefdbc7f469c9b849939947d1c82ce3aa61c5aaec47e53951c60416ad672d6e4\": container with ID starting with fefdbc7f469c9b849939947d1c82ce3aa61c5aaec47e53951c60416ad672d6e4 not found: ID does not exist"
Jul 28 22:34:19.727765 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.727743    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"]
Jul 28 22:34:19.734432 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.734413    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f8361843-55d97ccdb8-sf554"]
Jul 28 22:34:19.795385 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.795358    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:19.896029 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:19.896000    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:20.432046 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:20.432018    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f" path="/var/lib/kubelet/pods/5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f/volumes"
Jul 28 22:34:20.432342 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:20.432331    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f020a4b0-14b8-440a-ba36-4483960d9a28" path="/var/lib/kubelet/pods/f020a4b0-14b8-440a-ba36-4483960d9a28/volumes"
Jul 28 22:34:21.287518 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:21.287488    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:27.189292 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:27.189259    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:27.306037 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:27.306008    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:27.486571 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:27.486487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:27.753245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:27.753144    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:28.036604 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:28.036519    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:34:54.000573 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:34:54.000537    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:00.320437 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:00.320324    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:00.421642 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:00.421609    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:00.520811 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:00.520757    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:16.435436 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:16.435402    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:22.139398 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:22.139360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:22.207739 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:22.207705    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:22.277395 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:22.277365    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:22.378392 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:22.378361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:32.877229 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:32.877193    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:33.706695 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:33.706663    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:40.283210 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:40.283105    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:40.388466 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:40.388424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:40.491240 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:40.491205    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:40.585296 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:40.585219    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:56.408174 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:56.408140    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:56.504604 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:56.504574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:35:58.384182 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:35:58.384129    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:04.724352 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:04.724316    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:04.824440 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:04.824400    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:05.436695 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:05.436658    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:11.489958 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:11.489920    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:11.591881 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:11.591849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:11.731301 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:11.731268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:12.168900 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:12.168855    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:12.333436 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:12.333402    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:15.238555 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:15.238512    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:15.339162 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:15.339134    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:15.435733 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:15.435686    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:15.561754 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:15.561675    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:16.054789 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:16.054756    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:16.148613 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:16.148583    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:40.255455 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:40.255424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:40.950027 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:40.949992    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:41.047593 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:41.047555    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:41.382480 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:41.382452    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:47.656468 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:47.656436    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:47.734731 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:47.734703    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:48.043011 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.042929    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9"]
Jul 28 22:36:48.043274 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.043262    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f020a4b0-14b8-440a-ba36-4483960d9a28" containerName="revoke-keys"
Jul 28 22:36:48.043320 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.043276    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f020a4b0-14b8-440a-ba36-4483960d9a28" containerName="revoke-keys"
Jul 28 22:36:48.043320 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.043294    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f" containerName="maas-api"
Jul 28 22:36:48.043320 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.043300    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f" containerName="maas-api"
Jul 28 22:36:48.043417 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.043358    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b3d7f61-8af3-4b40-9ea6-8229ebcd2c0f" containerName="maas-api"
Jul 28 22:36:48.043417 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.043367    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f020a4b0-14b8-440a-ba36-4483960d9a28" containerName="revoke-keys"
Jul 28 22:36:48.046531 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.046515    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9"
Jul 28 22:36:48.049254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.049235    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:36:48.059544 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.059518    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9"]
Jul 28 22:36:48.129879 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.129849    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b1e19a7-e632-4705-92ec-ca73173bf4c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9\" (UID: \"7b1e19a7-e632-4705-92ec-ca73173bf4c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9"
Jul 28 22:36:48.231023 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.230992    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b1e19a7-e632-4705-92ec-ca73173bf4c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9\" (UID: \"7b1e19a7-e632-4705-92ec-ca73173bf4c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9"
Jul 28 22:36:48.231611 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.231589    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b1e19a7-e632-4705-92ec-ca73173bf4c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9\" (UID: \"7b1e19a7-e632-4705-92ec-ca73173bf4c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9"
Jul 28 22:36:48.356979 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.356903    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9"
Jul 28 22:36:48.492270 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.492247    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9"]
Jul 28 22:36:48.494231 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:36:48.494208    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b1e19a7_e632_4705_92ec_ca73173bf4c1.slice/crio-a7fb492ad427df829c267a9b0117e2e91fa02574e21403b31430c6d43ade88f3 WatchSource:0}: Error finding container a7fb492ad427df829c267a9b0117e2e91fa02574e21403b31430c6d43ade88f3: Status 404 returned error can't find the container with id a7fb492ad427df829c267a9b0117e2e91fa02574e21403b31430c6d43ade88f3
Jul 28 22:36:48.508329 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:48.508308    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9" event={"ID":"7b1e19a7-e632-4705-92ec-ca73173bf4c1","Type":"ContainerStarted","Data":"a7fb492ad427df829c267a9b0117e2e91fa02574e21403b31430c6d43ade88f3"}
Jul 28 22:36:49.515157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:49.515071    2583 generic.go:356] "Generic (PLEG): container finished" podID="7b1e19a7-e632-4705-92ec-ca73173bf4c1" containerID="c6aaf8393c2a24df22789e5120f7620e3e8c33fb0b8bad6718eecb8827ab9c82" exitCode=0
Jul 28 22:36:49.515157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:49.515134    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9" event={"ID":"7b1e19a7-e632-4705-92ec-ca73173bf4c1","Type":"ContainerDied","Data":"c6aaf8393c2a24df22789e5120f7620e3e8c33fb0b8bad6718eecb8827ab9c82"}
Jul 28 22:36:50.645853 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:50.645831    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9"
Jul 28 22:36:50.754854 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:50.754825    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b1e19a7-e632-4705-92ec-ca73173bf4c1-maas-api-service-ca\") pod \"7b1e19a7-e632-4705-92ec-ca73173bf4c1\" (UID: \"7b1e19a7-e632-4705-92ec-ca73173bf4c1\") "
Jul 28 22:36:50.755195 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:50.755174    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b1e19a7-e632-4705-92ec-ca73173bf4c1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7b1e19a7-e632-4705-92ec-ca73173bf4c1" (UID: "7b1e19a7-e632-4705-92ec-ca73173bf4c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:36:50.856453 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:50.856379    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b1e19a7-e632-4705-92ec-ca73173bf4c1-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:36:51.525982 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:51.525955    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9"
Jul 28 22:36:51.525982 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:51.525967    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9" event={"ID":"7b1e19a7-e632-4705-92ec-ca73173bf4c1","Type":"ContainerDied","Data":"a7fb492ad427df829c267a9b0117e2e91fa02574e21403b31430c6d43ade88f3"}
Jul 28 22:36:51.526193 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:51.526017    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7fb492ad427df829c267a9b0117e2e91fa02574e21403b31430c6d43ade88f3"
Jul 28 22:36:58.499302 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:58.499264    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:58.619992 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:58.619953    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:58.685169 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:58.685138    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9"]
Jul 28 22:36:58.690851 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:58.690826    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e64758c0-4wvr9"]
Jul 28 22:36:58.703908 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:58.703881    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:36:59.072350 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:36:59.072320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:00.431457 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:00.431412    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b1e19a7-e632-4705-92ec-ca73173bf4c1" path="/var/lib/kubelet/pods/7b1e19a7-e632-4705-92ec-ca73173bf4c1/volumes"
Jul 28 22:37:04.774034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:04.773996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:04.874185 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:04.874151    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:05.060842 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:05.060716    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:05.268203 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:05.268170    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:05.452442 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:05.452399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:05.581016 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:05.580983    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:05.675097 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:05.675066    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:11.607812 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:11.607692    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:11.741780 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:11.741733    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:11.828112 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:11.828080    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:11.967396 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:11.967365    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:18.020902 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:18.020859    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:18.122675 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:18.122644    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:24.997438 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:24.997403    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:26.091021 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:26.090990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:28.653938 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.653895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:28.754773 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.754742    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:28.797824 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.797785    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"]
Jul 28 22:37:28.798245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.798230    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b1e19a7-e632-4705-92ec-ca73173bf4c1" containerName="revoke-keys"
Jul 28 22:37:28.798307 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.798249    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b1e19a7-e632-4705-92ec-ca73173bf4c1" containerName="revoke-keys"
Jul 28 22:37:28.798369 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.798358    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b1e19a7-e632-4705-92ec-ca73173bf4c1" containerName="revoke-keys"
Jul 28 22:37:28.807099 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.807078    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:28.810289 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.810272    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tlhn2\""
Jul 28 22:37:28.811065 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.811044    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-01e687c7-serving-cert\""
Jul 28 22:37:28.820345 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.820326    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"]
Jul 28 22:37:28.838075 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.838052    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:28.985269 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.985182    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-maas-api-tls\") pod \"maas-api-e2e-mt-01e687c7-767596747d-4f7pb\" (UID: \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:28.985269 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.985243    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfvbb\" (UniqueName: \"kubernetes.io/projected/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-kube-api-access-rfvbb\") pod \"maas-api-e2e-mt-01e687c7-767596747d-4f7pb\" (UID: \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:28.985451 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:28.985347    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-openshift-service-ca\") pod \"maas-api-e2e-mt-01e687c7-767596747d-4f7pb\" (UID: \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:29.086890 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:29.086859    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-openshift-service-ca\") pod \"maas-api-e2e-mt-01e687c7-767596747d-4f7pb\" (UID: \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:29.087046 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:29.086936    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-maas-api-tls\") pod \"maas-api-e2e-mt-01e687c7-767596747d-4f7pb\" (UID: \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:29.087046 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:29.087005    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfvbb\" (UniqueName: \"kubernetes.io/projected/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-kube-api-access-rfvbb\") pod \"maas-api-e2e-mt-01e687c7-767596747d-4f7pb\" (UID: \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:29.087119 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:37:29.087082    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-serving-cert: secret "maas-api-e2e-mt-01e687c7-serving-cert" not found
Jul 28 22:37:29.087153 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:37:29.087145    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-maas-api-tls podName:d7737fd0-7d2f-4daf-aeb8-4eebf530c176 nodeName:}" failed. No retries permitted until 2026-07-28 22:37:29.587127573 +0000 UTC m=+1993.594993108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-maas-api-tls") pod "maas-api-e2e-mt-01e687c7-767596747d-4f7pb" (UID: "d7737fd0-7d2f-4daf-aeb8-4eebf530c176") : secret "maas-api-e2e-mt-01e687c7-serving-cert" not found
Jul 28 22:37:29.087576 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:29.087558    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-openshift-service-ca\") pod \"maas-api-e2e-mt-01e687c7-767596747d-4f7pb\" (UID: \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:29.095745 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:29.095721    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfvbb\" (UniqueName: \"kubernetes.io/projected/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-kube-api-access-rfvbb\") pod \"maas-api-e2e-mt-01e687c7-767596747d-4f7pb\" (UID: \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:29.357888 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:29.357812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:29.456438 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:29.456408    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:29.531857 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:29.531824    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:29.591306 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:29.591279    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-maas-api-tls\") pod \"maas-api-e2e-mt-01e687c7-767596747d-4f7pb\" (UID: \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:29.593586 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:29.593569    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-maas-api-tls\") pod \"maas-api-e2e-mt-01e687c7-767596747d-4f7pb\" (UID: \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:29.716518 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:29.716490    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:29.852375 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:29.852350    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"]
Jul 28 22:37:29.854074 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:37:29.854049    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7737fd0_7d2f_4daf_aeb8_4eebf530c176.slice/crio-c089bdd8e133c82be678ca5787502201f351c0b661f76a99b05c3f1dc49d334d WatchSource:0}: Error finding container c089bdd8e133c82be678ca5787502201f351c0b661f76a99b05c3f1dc49d334d: Status 404 returned error can't find the container with id c089bdd8e133c82be678ca5787502201f351c0b661f76a99b05c3f1dc49d334d
Jul 28 22:37:30.733858 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:30.733823    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb" event={"ID":"d7737fd0-7d2f-4daf-aeb8-4eebf530c176","Type":"ContainerStarted","Data":"eb2bb6358c1554a970869ea172123cd5372058ad60de33cced934dd84ab2d68d"}
Jul 28 22:37:30.734210 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:30.733875    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb" event={"ID":"d7737fd0-7d2f-4daf-aeb8-4eebf530c176","Type":"ContainerStarted","Data":"c089bdd8e133c82be678ca5787502201f351c0b661f76a99b05c3f1dc49d334d"}
Jul 28 22:37:30.734210 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:30.733903    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:30.753018 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:30.752973    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb" podStartSLOduration=2.2641389370000002 podStartE2EDuration="2.752959477s" totalImagesPullingTime="488.82054ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:37:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:37:30.750432594 +0000 UTC m=+1994.758298151" watchObservedRunningTime="2026-07-28 22:37:30.752959477 +0000 UTC m=+1994.760825049"
Jul 28 22:37:36.745065 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:36.745037    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:37:53.591095 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:53.591063    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:37:56.321719 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:37:56.321692    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:38:22.780534 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:22.780498    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5"]
Jul 28 22:38:22.783980 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:22.783964    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5"
Jul 28 22:38:22.786160 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:22.786139    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:38:22.790095 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:22.790069    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5"]
Jul 28 22:38:22.943363 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:22.943325    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bc9e51a-786d-4a66-bbd6-ee1a76241069-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-01e687c7-277b5\" (UID: \"3bc9e51a-786d-4a66-bbd6-ee1a76241069\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5"
Jul 28 22:38:23.044417 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:23.044343    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bc9e51a-786d-4a66-bbd6-ee1a76241069-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-01e687c7-277b5\" (UID: \"3bc9e51a-786d-4a66-bbd6-ee1a76241069\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5"
Jul 28 22:38:23.045005 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:23.044984    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bc9e51a-786d-4a66-bbd6-ee1a76241069-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-01e687c7-277b5\" (UID: \"3bc9e51a-786d-4a66-bbd6-ee1a76241069\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5"
Jul 28 22:38:23.096539 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:23.096517    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5"
Jul 28 22:38:23.226818 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:23.226779    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5"]
Jul 28 22:38:23.228784 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:38:23.228761    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc9e51a_786d_4a66_bbd6_ee1a76241069.slice/crio-76c8d0c39327c92917dd8f4ae3cd25137872d5d4f22f1ac6e49caec0d89c8e28 WatchSource:0}: Error finding container 76c8d0c39327c92917dd8f4ae3cd25137872d5d4f22f1ac6e49caec0d89c8e28: Status 404 returned error can't find the container with id 76c8d0c39327c92917dd8f4ae3cd25137872d5d4f22f1ac6e49caec0d89c8e28
Jul 28 22:38:24.025619 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:24.025589    2583 generic.go:356] "Generic (PLEG): container finished" podID="3bc9e51a-786d-4a66-bbd6-ee1a76241069" containerID="e2a55870f209d63701f0fbeaf3e48d44579849ce14eda0f34d34f1d19140b915" exitCode=0
Jul 28 22:38:24.025992 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:24.025641    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5" event={"ID":"3bc9e51a-786d-4a66-bbd6-ee1a76241069","Type":"ContainerDied","Data":"e2a55870f209d63701f0fbeaf3e48d44579849ce14eda0f34d34f1d19140b915"}
Jul 28 22:38:24.025992 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:24.025708    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5" event={"ID":"3bc9e51a-786d-4a66-bbd6-ee1a76241069","Type":"ContainerStarted","Data":"76c8d0c39327c92917dd8f4ae3cd25137872d5d4f22f1ac6e49caec0d89c8e28"}
Jul 28 22:38:25.155899 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:25.155874    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5"
Jul 28 22:38:25.263330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:25.263296    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bc9e51a-786d-4a66-bbd6-ee1a76241069-maas-api-service-ca\") pod \"3bc9e51a-786d-4a66-bbd6-ee1a76241069\" (UID: \"3bc9e51a-786d-4a66-bbd6-ee1a76241069\") "
Jul 28 22:38:25.263668 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:25.263649    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bc9e51a-786d-4a66-bbd6-ee1a76241069-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3bc9e51a-786d-4a66-bbd6-ee1a76241069" (UID: "3bc9e51a-786d-4a66-bbd6-ee1a76241069"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:38:25.364504 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:25.364481    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bc9e51a-786d-4a66-bbd6-ee1a76241069-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:38:26.037786 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:26.037748    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5" event={"ID":"3bc9e51a-786d-4a66-bbd6-ee1a76241069","Type":"ContainerDied","Data":"76c8d0c39327c92917dd8f4ae3cd25137872d5d4f22f1ac6e49caec0d89c8e28"}
Jul 28 22:38:26.037786 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:26.037775    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5"
Jul 28 22:38:26.038009 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:26.037825    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76c8d0c39327c92917dd8f4ae3cd25137872d5d4f22f1ac6e49caec0d89c8e28"
Jul 28 22:38:38.297713 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.297638    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"]
Jul 28 22:38:38.298153 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.297846    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb" podUID="d7737fd0-7d2f-4daf-aeb8-4eebf530c176" containerName="maas-api" containerID="cri-o://eb2bb6358c1554a970869ea172123cd5372058ad60de33cced934dd84ab2d68d" gracePeriod=30
Jul 28 22:38:38.332371 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.332346    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:38:38.404261 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.404224    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:38:38.462741 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.462705    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:38:38.547834 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.547755    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:38:38.566056 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.565969    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:38:38.572917 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.572893    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfvbb\" (UniqueName: \"kubernetes.io/projected/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-kube-api-access-rfvbb\") pod \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\" (UID: \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\") "
Jul 28 22:38:38.573079 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.573063    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-maas-api-tls\") pod \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\" (UID: \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\") "
Jul 28 22:38:38.573148 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.573109    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-openshift-service-ca\") pod \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\" (UID: \"d7737fd0-7d2f-4daf-aeb8-4eebf530c176\") "
Jul 28 22:38:38.573460 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.573432    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7737fd0-7d2f-4daf-aeb8-4eebf530c176" (UID: "d7737fd0-7d2f-4daf-aeb8-4eebf530c176"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:38:38.575231 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.575200    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-kube-api-access-rfvbb" (OuterVolumeSpecName: "kube-api-access-rfvbb") pod "d7737fd0-7d2f-4daf-aeb8-4eebf530c176" (UID: "d7737fd0-7d2f-4daf-aeb8-4eebf530c176"). InnerVolumeSpecName "kube-api-access-rfvbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:38:38.575385 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.575208    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7737fd0-7d2f-4daf-aeb8-4eebf530c176" (UID: "d7737fd0-7d2f-4daf-aeb8-4eebf530c176"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:38:38.578208 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.578187    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5"]
Jul 28 22:38:38.581805 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.581770    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-01e687c7-277b5"]
Jul 28 22:38:38.673791 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.673761    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-maas-api-tls\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:38:38.673791 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.673785    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-openshift-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:38:38.673990 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:38.673813    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfvbb\" (UniqueName: \"kubernetes.io/projected/d7737fd0-7d2f-4daf-aeb8-4eebf530c176-kube-api-access-rfvbb\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:38:39.108134 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:39.108102    2583 generic.go:356] "Generic (PLEG): container finished" podID="d7737fd0-7d2f-4daf-aeb8-4eebf530c176" containerID="eb2bb6358c1554a970869ea172123cd5372058ad60de33cced934dd84ab2d68d" exitCode=0
Jul 28 22:38:39.108298 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:39.108143    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"
Jul 28 22:38:39.108298 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:39.108166    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb" event={"ID":"d7737fd0-7d2f-4daf-aeb8-4eebf530c176","Type":"ContainerDied","Data":"eb2bb6358c1554a970869ea172123cd5372058ad60de33cced934dd84ab2d68d"}
Jul 28 22:38:39.108298 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:39.108205    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb" event={"ID":"d7737fd0-7d2f-4daf-aeb8-4eebf530c176","Type":"ContainerDied","Data":"c089bdd8e133c82be678ca5787502201f351c0b661f76a99b05c3f1dc49d334d"}
Jul 28 22:38:39.108298 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:39.108229    2583 scope.go:117] "RemoveContainer" containerID="eb2bb6358c1554a970869ea172123cd5372058ad60de33cced934dd84ab2d68d"
Jul 28 22:38:39.117440 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:39.117425    2583 scope.go:117] "RemoveContainer" containerID="eb2bb6358c1554a970869ea172123cd5372058ad60de33cced934dd84ab2d68d"
Jul 28 22:38:39.117692 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:38:39.117670    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb2bb6358c1554a970869ea172123cd5372058ad60de33cced934dd84ab2d68d\": container with ID starting with eb2bb6358c1554a970869ea172123cd5372058ad60de33cced934dd84ab2d68d not found: ID does not exist" containerID="eb2bb6358c1554a970869ea172123cd5372058ad60de33cced934dd84ab2d68d"
Jul 28 22:38:39.117752 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:39.117700    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb2bb6358c1554a970869ea172123cd5372058ad60de33cced934dd84ab2d68d"} err="failed to get container status \"eb2bb6358c1554a970869ea172123cd5372058ad60de33cced934dd84ab2d68d\": rpc error: code = NotFound desc = could not find container \"eb2bb6358c1554a970869ea172123cd5372058ad60de33cced934dd84ab2d68d\": container with ID starting with eb2bb6358c1554a970869ea172123cd5372058ad60de33cced934dd84ab2d68d not found: ID does not exist"
Jul 28 22:38:39.127674 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:39.127654    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"]
Jul 28 22:38:39.131276 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:39.131253    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-01e687c7-767596747d-4f7pb"]
Jul 28 22:38:40.431050 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:40.431012    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bc9e51a-786d-4a66-bbd6-ee1a76241069" path="/var/lib/kubelet/pods/3bc9e51a-786d-4a66-bbd6-ee1a76241069/volumes"
Jul 28 22:38:40.431423 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:40.431388    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7737fd0-7d2f-4daf-aeb8-4eebf530c176" path="/var/lib/kubelet/pods/d7737fd0-7d2f-4daf-aeb8-4eebf530c176/volumes"
Jul 28 22:38:41.965966 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:41.965937    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:38:46.996417 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:38:46.996383    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:16.549100 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:16.549067    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:39:16.563154 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:16.563131    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:39:16.622136 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:16.622099    2583 scope.go:117] "RemoveContainer" containerID="5f1be7895ee99f053589efc549379a7a8e21bd9f3c0c0018278984f480a63eee"
Jul 28 22:39:16.630527 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:16.630509    2583 scope.go:117] "RemoveContainer" containerID="c43e50ada620b7c23905f7bd226c260a96b25645c288992a32a8858629ee806d"
Jul 28 22:39:24.089110 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:24.089074    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:24.306718 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:24.306693    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:30.314212 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:30.314177    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:30.406148 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:30.406114    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:30.666060 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:30.666030    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:30.966726 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:30.966643    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:32.012989 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:32.012957    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:32.106998 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:32.106966    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:32.208269 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:32.208235    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:32.959087 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:32.959015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:33.077437 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:33.077406    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:33.201818 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:33.201755    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:34.549249 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:34.549219    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:40.401214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:40.401176    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:40.503939 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:40.503905    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:41.084504 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:41.084469    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:47.229583 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:47.229551    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:47.489490 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:47.489407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:47.676077 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:47.676023    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:58.012251 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:58.012218    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:39:58.356092 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:39:58.356005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:16.658283 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:16.658210    2583 scope.go:117] "RemoveContainer" containerID="c0ca5c3a0ef542f7be4bbb91db8a6a9d82624d154a37ff1ce871eccdb89fe4fa"
Jul 28 22:40:16.666636 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:16.666618    2583 scope.go:117] "RemoveContainer" containerID="1b1b55927890ad8f696547b48a4a1bb123c4ad518c45dadbcf11689542fedbcd"
Jul 28 22:40:19.369605 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:19.369573    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:19.651771 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:19.651686    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:25.875244 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:25.875211    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:25.975850 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:25.975790    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:26.259035 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:26.258950    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:26.645075 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:26.645044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:26.865374 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:26.865341    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:29.418854 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:29.418789    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:29.511303 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:29.511273    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:29.579209 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:29.579173    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:29.895880 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:29.895845    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:30.072707 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:30.072668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:30.208865 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:30.208763    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:34.394289 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:34.394254    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:54.450690 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:54.450654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:54.802724 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:54.802638    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:55.083278 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.083194    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"]
Jul 28 22:40:55.083760 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.083745    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7737fd0-7d2f-4daf-aeb8-4eebf530c176" containerName="maas-api"
Jul 28 22:40:55.083828 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.083764    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7737fd0-7d2f-4daf-aeb8-4eebf530c176" containerName="maas-api"
Jul 28 22:40:55.083828 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.083779    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bc9e51a-786d-4a66-bbd6-ee1a76241069" containerName="revoke-keys"
Jul 28 22:40:55.083828 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.083806    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bc9e51a-786d-4a66-bbd6-ee1a76241069" containerName="revoke-keys"
Jul 28 22:40:55.083926 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.083918    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7737fd0-7d2f-4daf-aeb8-4eebf530c176" containerName="maas-api"
Jul 28 22:40:55.083960 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.083933    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bc9e51a-786d-4a66-bbd6-ee1a76241069" containerName="revoke-keys"
Jul 28 22:40:55.087381 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.087360    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.089429 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.089408    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 22:40:55.089429 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.089423    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-8a6fb9-openshift-default-dockercfg-xr6pp\""
Jul 28 22:40:55.098491 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.098461    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"]
Jul 28 22:40:55.116159 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.116120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:55.204755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.204720    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-istio-envoy\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.204755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.204767    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg2tb\" (UniqueName: \"kubernetes.io/projected/331b742f-314f-4c56-8679-0f5d59181a2d-kube-api-access-qg2tb\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.205059 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.204816    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-workload-certs\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.205059 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.204838    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/331b742f-314f-4c56-8679-0f5d59181a2d-istio-podinfo\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.205059 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.204861    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-credential-socket\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.205059 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.204886    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-istio-data\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.205059 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.204901    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/331b742f-314f-4c56-8679-0f5d59181a2d-istiod-ca-cert\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.205059 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.204970    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-workload-socket\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.205059 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.205005    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/331b742f-314f-4c56-8679-0f5d59181a2d-istio-token\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.305681 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.305637    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-istio-envoy\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.305914 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.305693    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg2tb\" (UniqueName: \"kubernetes.io/projected/331b742f-314f-4c56-8679-0f5d59181a2d-kube-api-access-qg2tb\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.305914 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.305723    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-workload-certs\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.305914 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.305743    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/331b742f-314f-4c56-8679-0f5d59181a2d-istio-podinfo\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.305914 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.305769    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-credential-socket\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.305914 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.305833    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-istio-data\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.305914 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.305862    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/331b742f-314f-4c56-8679-0f5d59181a2d-istiod-ca-cert\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.305914 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.305906    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-workload-socket\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.306279 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.305930    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/331b742f-314f-4c56-8679-0f5d59181a2d-istio-token\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.306406 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.306375    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-workload-certs\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.306553 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.306531    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-workload-socket\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.306764 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.306741    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/331b742f-314f-4c56-8679-0f5d59181a2d-istiod-ca-cert\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.306866 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.306760    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-credential-socket\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.306956 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.306939    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-istio-data\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.308207 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.308191    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-istio-envoy\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.308390 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.308371    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/331b742f-314f-4c56-8679-0f5d59181a2d-istio-podinfo\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.316984 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.316963    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg2tb\" (UniqueName: \"kubernetes.io/projected/331b742f-314f-4c56-8679-0f5d59181a2d-kube-api-access-qg2tb\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.317161 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.317144    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/331b742f-314f-4c56-8679-0f5d59181a2d-istio-token\") pod \"e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") " pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.331273 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.331244    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:40:55.399653 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.399625    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:55.541448 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.541332    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"]
Jul 28 22:40:55.543907 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:40:55.543880    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod331b742f_314f_4c56_8679_0f5d59181a2d.slice/crio-3c368ead96b253eaefd5253466af25a5c5adfa54c939e4fdac1c6e9966402159 WatchSource:0}: Error finding container 3c368ead96b253eaefd5253466af25a5c5adfa54c939e4fdac1c6e9966402159: Status 404 returned error can't find the container with id 3c368ead96b253eaefd5253466af25a5c5adfa54c939e4fdac1c6e9966402159
Jul 28 22:40:55.545906 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.545888    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:40:55.831338 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:55.831254    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c" event={"ID":"331b742f-314f-4c56-8679-0f5d59181a2d","Type":"ContainerStarted","Data":"3c368ead96b253eaefd5253466af25a5c5adfa54c939e4fdac1c6e9966402159"}
Jul 28 22:40:58.291856 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:58.291818    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:40:58.292107 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:58.291893    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:40:58.292107 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:58.291923    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:40:58.851747 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:58.851709    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c" event={"ID":"331b742f-314f-4c56-8679-0f5d59181a2d","Type":"ContainerStarted","Data":"4fbbe935503ae3ff4216e61714492fd1afef019035a7e2d60988633dc411f830"}
Jul 28 22:40:58.871536 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:58.871480    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c" podStartSLOduration=1.126088494 podStartE2EDuration="3.871465289s" totalImagesPullingTime="2.745376795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:40:58.868359438 +0000 UTC m=+2202.876225015" watchObservedRunningTime="2026-07-28 22:40:58.871465289 +0000 UTC m=+2202.879330844"
Jul 28 22:40:59.400622 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:59.400582    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:59.405556 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:59.405527    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:59.857548 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:59.857457    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:40:59.858350 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:40:59.858330    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:41:00.802142 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:00.802106    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:41:00.884252 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:00.884224    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:41:00.953269 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:00.953218    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:41:01.113806 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:01.113746    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:41:01.562230 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:01.562125    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:41:01.664011 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:01.663974    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:41:13.933555 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:13.933522    2583 generic.go:356] "Generic (PLEG): container finished" podID="331b742f-314f-4c56-8679-0f5d59181a2d" containerID="4fbbe935503ae3ff4216e61714492fd1afef019035a7e2d60988633dc411f830" exitCode=0
Jul 28 22:41:13.934003 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:13.933590    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c" event={"ID":"331b742f-314f-4c56-8679-0f5d59181a2d","Type":"ContainerDied","Data":"4fbbe935503ae3ff4216e61714492fd1afef019035a7e2d60988633dc411f830"}
Jul 28 22:41:13.934057 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:13.934043    2583 scope.go:117] "RemoveContainer" containerID="4fbbe935503ae3ff4216e61714492fd1afef019035a7e2d60988633dc411f830"
Jul 28 22:41:13.935091 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:13.935065    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:41:13.935154 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:13.935137    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:41:13.935198 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:13.935168    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:41:14.857699 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:14.857591    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:41:14.956097 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:14.956054    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c" event={"ID":"331b742f-314f-4c56-8679-0f5d59181a2d","Type":"ContainerStarted","Data":"0213e846559514ed7a8265c074e3fd3e0af97b3961e06e02bd3ad529c6e8c420"}
Jul 28 22:41:15.400569 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:15.400531    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:41:15.401455 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:15.401428    2583 patch_prober.go:28] interesting pod/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 28 22:41:15.401574 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:15.401496    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c" podUID="331b742f-314f-4c56-8679-0f5d59181a2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 28 22:41:16.400728 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:16.400693    2583 patch_prober.go:28] interesting pod/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 28 22:41:16.401160 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:16.400747    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c" podUID="331b742f-314f-4c56-8679-0f5d59181a2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 28 22:41:17.400697 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:17.400642    2583 patch_prober.go:28] interesting pod/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 28 22:41:17.400910 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:17.400715    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c" podUID="331b742f-314f-4c56-8679-0f5d59181a2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 28 22:41:18.400453 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:18.400415    2583 patch_prober.go:28] interesting pod/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 28 22:41:18.400645 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:18.400482    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c" podUID="331b742f-314f-4c56-8679-0f5d59181a2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 28 22:41:19.404394 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:19.404366    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:41:19.404854 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:19.404633    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:41:19.405303 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:19.405288    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:41:25.792596 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:25.792549    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:41:30.132382 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:30.132343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:41:55.292891 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.292857    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"]
Jul 28 22:41:55.296447 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.296426    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.298518 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.298486    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8a6fb9\"/\"kube-root-ca.crt\""
Jul 28 22:41:55.299126 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.299107    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8a6fb9\"/\"default-dockercfg-crtx7\""
Jul 28 22:41:55.299213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.299176    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8a6fb9\"/\"test-model-8a6fb9-kserve-self-signed-certs\""
Jul 28 22:41:55.299276 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.299226    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8a6fb9\"/\"openshift-service-ca.crt\""
Jul 28 22:41:55.306256 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.306233    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"]
Jul 28 22:41:55.316095 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.316068    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:41:55.352286 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.352264    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b856e28-726f-42dc-b287-ab9b6da2c8c7-tls-certs\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.352381 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.352293    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-tmp-dir\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.352381 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.352312    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-home\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.352381 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.352334    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-model-cache\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.352381 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.352349    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwp9g\" (UniqueName: \"kubernetes.io/projected/5b856e28-726f-42dc-b287-ab9b6da2c8c7-kube-api-access-pwp9g\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.352517 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.352425    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-dshm\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.453471 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.453445    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-dshm\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.453595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.453492    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b856e28-726f-42dc-b287-ab9b6da2c8c7-tls-certs\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.453595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.453512    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-tmp-dir\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.453595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.453528    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-home\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.453595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.453547    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-model-cache\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.453595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.453563    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwp9g\" (UniqueName: \"kubernetes.io/projected/5b856e28-726f-42dc-b287-ab9b6da2c8c7-kube-api-access-pwp9g\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.453936 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.453915    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-tmp-dir\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.454001 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.453962    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-home\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.454001 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.453988    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-model-cache\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.455641 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.455614    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-dshm\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.456084 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.456064    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b856e28-726f-42dc-b287-ab9b6da2c8c7-tls-certs\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.460430 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.460412    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwp9g\" (UniqueName: \"kubernetes.io/projected/5b856e28-726f-42dc-b287-ab9b6da2c8c7-kube-api-access-pwp9g\") pod \"test-model-8a6fb9-kserve-89cf88485-czzjq\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") " pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.606654 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.606579    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:41:55.739196 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:55.739172    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"]
Jul 28 22:41:55.740964 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:41:55.740939    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b856e28_726f_42dc_b287_ab9b6da2c8c7.slice/crio-58e3d1ad2bcbbe64d9ce07fcb5922dbcdc39da2faaeb0cfc3ad478f6fd879474 WatchSource:0}: Error finding container 58e3d1ad2bcbbe64d9ce07fcb5922dbcdc39da2faaeb0cfc3ad478f6fd879474: Status 404 returned error can't find the container with id 58e3d1ad2bcbbe64d9ce07fcb5922dbcdc39da2faaeb0cfc3ad478f6fd879474
Jul 28 22:41:56.175147 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:56.175115    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq" event={"ID":"5b856e28-726f-42dc-b287-ab9b6da2c8c7","Type":"ContainerStarted","Data":"33df12f68dad6ffe45af89ce49085d24601cecbfee01960fe1ee496e64c384aa"}
Jul 28 22:41:56.175147 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:56.175151    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq" event={"ID":"5b856e28-726f-42dc-b287-ab9b6da2c8c7","Type":"ContainerStarted","Data":"58e3d1ad2bcbbe64d9ce07fcb5922dbcdc39da2faaeb0cfc3ad478f6fd879474"}
Jul 28 22:41:56.191301 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:41:56.191255    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq" podStartSLOduration=1.19124038 podStartE2EDuration="1.19124038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:41:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:41:56.188941818 +0000 UTC m=+2260.196807392" watchObservedRunningTime="2026-07-28 22:41:56.19124038 +0000 UTC m=+2260.199105937"
Jul 28 22:42:05.607051 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:05.607025    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:42:05.607404 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:05.607197    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:42:05.619718 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:05.619698    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:42:06.242011 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:06.241986    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:42:17.989266 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:17.989232    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:42:18.087112 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:18.087077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:42:45.518533 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:45.518493    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:42:50.608410 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:50.608378    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:42:51.500731 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:51.500700    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"]
Jul 28 22:42:51.519576 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:51.519543    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:42:51.663815 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:51.663773    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv"]
Jul 28 22:42:51.667073 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:51.667059    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv"
Jul 28 22:42:51.668904 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:51.668882    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:42:51.673395 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:51.673361    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv"]
Jul 28 22:42:51.751822 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:51.751722    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8975eaa8-887b-4c0d-89c6-dbebc5f0fb54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-00957a-rwndv\" (UID: \"8975eaa8-887b-4c0d-89c6-dbebc5f0fb54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv"
Jul 28 22:42:51.852560 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:51.852527    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8975eaa8-887b-4c0d-89c6-dbebc5f0fb54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-00957a-rwndv\" (UID: \"8975eaa8-887b-4c0d-89c6-dbebc5f0fb54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv"
Jul 28 22:42:51.853162 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:51.853143    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8975eaa8-887b-4c0d-89c6-dbebc5f0fb54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-00957a-rwndv\" (UID: \"8975eaa8-887b-4c0d-89c6-dbebc5f0fb54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv"
Jul 28 22:42:51.978973 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:51.978945    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv"
Jul 28 22:42:52.115633 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:52.115607    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv"]
Jul 28 22:42:52.117476 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:42:52.117452    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8975eaa8_887b_4c0d_89c6_dbebc5f0fb54.slice/crio-235e9df1fb5c22c2886a19896d7043b4199f87d2095c082f5a760a135a6c2ca2 WatchSource:0}: Error finding container 235e9df1fb5c22c2886a19896d7043b4199f87d2095c082f5a760a135a6c2ca2: Status 404 returned error can't find the container with id 235e9df1fb5c22c2886a19896d7043b4199f87d2095c082f5a760a135a6c2ca2
Jul 28 22:42:52.477779 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:52.477742    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv" event={"ID":"8975eaa8-887b-4c0d-89c6-dbebc5f0fb54","Type":"ContainerStarted","Data":"f779c90d0b1cdbf9911867db59bc9d2d4301a44b721dc6127cc122462f6367a8"}
Jul 28 22:42:52.477910 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:52.477823    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv" event={"ID":"8975eaa8-887b-4c0d-89c6-dbebc5f0fb54","Type":"ContainerStarted","Data":"235e9df1fb5c22c2886a19896d7043b4199f87d2095c082f5a760a135a6c2ca2"}
Jul 28 22:42:52.493404 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:52.493364    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv" podStartSLOduration=1.493350945 podStartE2EDuration="1.493350945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:42:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:42:52.49083598 +0000 UTC m=+2316.498701544" watchObservedRunningTime="2026-07-28 22:42:52.493350945 +0000 UTC m=+2316.501216514"
Jul 28 22:42:53.485037 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:53.484999    2583 generic.go:356] "Generic (PLEG): container finished" podID="8975eaa8-887b-4c0d-89c6-dbebc5f0fb54" containerID="f779c90d0b1cdbf9911867db59bc9d2d4301a44b721dc6127cc122462f6367a8" exitCode=0
Jul 28 22:42:53.485435 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:53.485079    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv" event={"ID":"8975eaa8-887b-4c0d-89c6-dbebc5f0fb54","Type":"ContainerDied","Data":"f779c90d0b1cdbf9911867db59bc9d2d4301a44b721dc6127cc122462f6367a8"}
Jul 28 22:42:54.622147 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:54.622124    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv"
Jul 28 22:42:54.779150 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:54.779052    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8975eaa8-887b-4c0d-89c6-dbebc5f0fb54-maas-api-service-ca\") pod \"8975eaa8-887b-4c0d-89c6-dbebc5f0fb54\" (UID: \"8975eaa8-887b-4c0d-89c6-dbebc5f0fb54\") "
Jul 28 22:42:54.779408 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:54.779386    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8975eaa8-887b-4c0d-89c6-dbebc5f0fb54-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8975eaa8-887b-4c0d-89c6-dbebc5f0fb54" (UID: "8975eaa8-887b-4c0d-89c6-dbebc5f0fb54"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:42:54.880552 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:54.880516    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8975eaa8-887b-4c0d-89c6-dbebc5f0fb54-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:42:55.122607 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:55.122574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:42:55.497098 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:55.497007    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv"
Jul 28 22:42:55.497098 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:55.497030    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv" event={"ID":"8975eaa8-887b-4c0d-89c6-dbebc5f0fb54","Type":"ContainerDied","Data":"235e9df1fb5c22c2886a19896d7043b4199f87d2095c082f5a760a135a6c2ca2"}
Jul 28 22:42:55.497098 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:42:55.497075    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="235e9df1fb5c22c2886a19896d7043b4199f87d2095c082f5a760a135a6c2ca2"
Jul 28 22:43:02.227599 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:02.227564    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:02.295752 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:02.295723    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:02.363366 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:02.363335    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:02.415329 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:02.415299    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv"]
Jul 28 22:43:02.418972 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:02.418948    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-00957a-rwndv"]
Jul 28 22:43:02.432137 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:02.432100    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8975eaa8-887b-4c0d-89c6-dbebc5f0fb54" path="/var/lib/kubelet/pods/8975eaa8-887b-4c0d-89c6-dbebc5f0fb54/volumes"
Jul 28 22:43:02.434016 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:02.433983    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:02.812607 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:02.812560    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:06.519756 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.519699    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq" podUID="5b856e28-726f-42dc-b287-ab9b6da2c8c7" containerName="main" containerID="cri-o://33df12f68dad6ffe45af89ce49085d24601cecbfee01960fe1ee496e64c384aa" gracePeriod=45
Jul 28 22:43:06.772762 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.772707    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:43:06.888102 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.888072    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-tmp-dir\") pod \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") "
Jul 28 22:43:06.888279 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.888124    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b856e28-726f-42dc-b287-ab9b6da2c8c7-tls-certs\") pod \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") "
Jul 28 22:43:06.888279 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.888144    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-dshm\") pod \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") "
Jul 28 22:43:06.888279 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.888170    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-model-cache\") pod \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") "
Jul 28 22:43:06.888279 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.888190    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwp9g\" (UniqueName: \"kubernetes.io/projected/5b856e28-726f-42dc-b287-ab9b6da2c8c7-kube-api-access-pwp9g\") pod \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") "
Jul 28 22:43:06.888279 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.888215    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-home\") pod \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\" (UID: \"5b856e28-726f-42dc-b287-ab9b6da2c8c7\") "
Jul 28 22:43:06.888537 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.888326    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5b856e28-726f-42dc-b287-ab9b6da2c8c7" (UID: "5b856e28-726f-42dc-b287-ab9b6da2c8c7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:06.888537 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.888443    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-model-cache" (OuterVolumeSpecName: "model-cache") pod "5b856e28-726f-42dc-b287-ab9b6da2c8c7" (UID: "5b856e28-726f-42dc-b287-ab9b6da2c8c7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:06.888537 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.888514    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-tmp-dir\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:06.888537 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.888505    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-home" (OuterVolumeSpecName: "home") pod "5b856e28-726f-42dc-b287-ab9b6da2c8c7" (UID: "5b856e28-726f-42dc-b287-ab9b6da2c8c7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:06.890449 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.890418    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-dshm" (OuterVolumeSpecName: "dshm") pod "5b856e28-726f-42dc-b287-ab9b6da2c8c7" (UID: "5b856e28-726f-42dc-b287-ab9b6da2c8c7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:06.890551 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.890494    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b856e28-726f-42dc-b287-ab9b6da2c8c7-kube-api-access-pwp9g" (OuterVolumeSpecName: "kube-api-access-pwp9g") pod "5b856e28-726f-42dc-b287-ab9b6da2c8c7" (UID: "5b856e28-726f-42dc-b287-ab9b6da2c8c7"). InnerVolumeSpecName "kube-api-access-pwp9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:43:06.890551 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.890499    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b856e28-726f-42dc-b287-ab9b6da2c8c7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5b856e28-726f-42dc-b287-ab9b6da2c8c7" (UID: "5b856e28-726f-42dc-b287-ab9b6da2c8c7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:43:06.989605 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.989575    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b856e28-726f-42dc-b287-ab9b6da2c8c7-tls-certs\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:06.989605 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.989600    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-dshm\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:06.989605 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.989608    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-model-cache\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:06.989821 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.989619    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwp9g\" (UniqueName: \"kubernetes.io/projected/5b856e28-726f-42dc-b287-ab9b6da2c8c7-kube-api-access-pwp9g\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:06.989821 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:06.989628    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b856e28-726f-42dc-b287-ab9b6da2c8c7-home\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:07.565870 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:07.565836    2583 generic.go:356] "Generic (PLEG): container finished" podID="5b856e28-726f-42dc-b287-ab9b6da2c8c7" containerID="33df12f68dad6ffe45af89ce49085d24601cecbfee01960fe1ee496e64c384aa" exitCode=0
Jul 28 22:43:07.566368 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:07.565890    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"
Jul 28 22:43:07.566368 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:07.565928    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq" event={"ID":"5b856e28-726f-42dc-b287-ab9b6da2c8c7","Type":"ContainerDied","Data":"33df12f68dad6ffe45af89ce49085d24601cecbfee01960fe1ee496e64c384aa"}
Jul 28 22:43:07.566368 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:07.565983    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq" event={"ID":"5b856e28-726f-42dc-b287-ab9b6da2c8c7","Type":"ContainerDied","Data":"58e3d1ad2bcbbe64d9ce07fcb5922dbcdc39da2faaeb0cfc3ad478f6fd879474"}
Jul 28 22:43:07.566368 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:07.566011    2583 scope.go:117] "RemoveContainer" containerID="33df12f68dad6ffe45af89ce49085d24601cecbfee01960fe1ee496e64c384aa"
Jul 28 22:43:07.576414 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:07.576396    2583 scope.go:117] "RemoveContainer" containerID="33df12f68dad6ffe45af89ce49085d24601cecbfee01960fe1ee496e64c384aa"
Jul 28 22:43:07.576664 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:43:07.576643    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33df12f68dad6ffe45af89ce49085d24601cecbfee01960fe1ee496e64c384aa\": container with ID starting with 33df12f68dad6ffe45af89ce49085d24601cecbfee01960fe1ee496e64c384aa not found: ID does not exist" containerID="33df12f68dad6ffe45af89ce49085d24601cecbfee01960fe1ee496e64c384aa"
Jul 28 22:43:07.576741 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:07.576670    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33df12f68dad6ffe45af89ce49085d24601cecbfee01960fe1ee496e64c384aa"} err="failed to get container status \"33df12f68dad6ffe45af89ce49085d24601cecbfee01960fe1ee496e64c384aa\": rpc error: code = NotFound desc = could not find container \"33df12f68dad6ffe45af89ce49085d24601cecbfee01960fe1ee496e64c384aa\": container with ID starting with 33df12f68dad6ffe45af89ce49085d24601cecbfee01960fe1ee496e64c384aa not found: ID does not exist"
Jul 28 22:43:07.588408 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:07.588387    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"]
Jul 28 22:43:07.594050 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:07.594031    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-8a6fb9/test-model-8a6fb9-kserve-89cf88485-czzjq"]
Jul 28 22:43:08.431296 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:08.431225    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b856e28-726f-42dc-b287-ab9b6da2c8c7" path="/var/lib/kubelet/pods/5b856e28-726f-42dc-b287-ab9b6da2c8c7/volumes"
Jul 28 22:43:08.718498 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:08.718409    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:08.826662 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:08.826627    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:09.011864 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.011758    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:09.258824 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.258759    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:09.373597 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.373565    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf"]
Jul 28 22:43:09.373963 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.373950    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8975eaa8-887b-4c0d-89c6-dbebc5f0fb54" containerName="revoke-keys"
Jul 28 22:43:09.374011 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.373964    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8975eaa8-887b-4c0d-89c6-dbebc5f0fb54" containerName="revoke-keys"
Jul 28 22:43:09.374011 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.373982    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b856e28-726f-42dc-b287-ab9b6da2c8c7" containerName="main"
Jul 28 22:43:09.374011 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.373988    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b856e28-726f-42dc-b287-ab9b6da2c8c7" containerName="main"
Jul 28 22:43:09.374113 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.374044    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b856e28-726f-42dc-b287-ab9b6da2c8c7" containerName="main"
Jul 28 22:43:09.374113 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.374052    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8975eaa8-887b-4c0d-89c6-dbebc5f0fb54" containerName="revoke-keys"
Jul 28 22:43:09.378413 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.378399    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf"
Jul 28 22:43:09.380427 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.380403    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:43:09.383585 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.383562    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf"]
Jul 28 22:43:09.512212 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.512182    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dd3ea32-301f-4c92-abce-5c667f959d12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf\" (UID: \"2dd3ea32-301f-4c92-abce-5c667f959d12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf"
Jul 28 22:43:09.613151 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.613124    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dd3ea32-301f-4c92-abce-5c667f959d12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf\" (UID: \"2dd3ea32-301f-4c92-abce-5c667f959d12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf"
Jul 28 22:43:09.613733 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.613715    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dd3ea32-301f-4c92-abce-5c667f959d12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf\" (UID: \"2dd3ea32-301f-4c92-abce-5c667f959d12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf"
Jul 28 22:43:09.690649 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.690559    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf"
Jul 28 22:43:09.820456 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:09.820431    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf"]
Jul 28 22:43:09.822607 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:43:09.822584    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dd3ea32_301f_4c92_abce_5c667f959d12.slice/crio-3da81fb394487aa3e7fe7fc6a606b47f99ec12e874af5ff89a370750c3d102ca WatchSource:0}: Error finding container 3da81fb394487aa3e7fe7fc6a606b47f99ec12e874af5ff89a370750c3d102ca: Status 404 returned error can't find the container with id 3da81fb394487aa3e7fe7fc6a606b47f99ec12e874af5ff89a370750c3d102ca
Jul 28 22:43:10.585304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:10.585267    2583 generic.go:356] "Generic (PLEG): container finished" podID="2dd3ea32-301f-4c92-abce-5c667f959d12" containerID="e5d909733603c728eaa78c5c5ffcbbda52dff2d2d58a90b11ca5fef54a70d45c" exitCode=0
Jul 28 22:43:10.585479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:10.585315    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf" event={"ID":"2dd3ea32-301f-4c92-abce-5c667f959d12","Type":"ContainerDied","Data":"e5d909733603c728eaa78c5c5ffcbbda52dff2d2d58a90b11ca5fef54a70d45c"}
Jul 28 22:43:10.585479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:10.585380    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf" event={"ID":"2dd3ea32-301f-4c92-abce-5c667f959d12","Type":"ContainerStarted","Data":"3da81fb394487aa3e7fe7fc6a606b47f99ec12e874af5ff89a370750c3d102ca"}
Jul 28 22:43:11.716637 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:11.716611    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf"
Jul 28 22:43:11.731595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:11.731554    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dd3ea32-301f-4c92-abce-5c667f959d12-maas-api-service-ca\") pod \"2dd3ea32-301f-4c92-abce-5c667f959d12\" (UID: \"2dd3ea32-301f-4c92-abce-5c667f959d12\") "
Jul 28 22:43:11.732021 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:11.731984    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dd3ea32-301f-4c92-abce-5c667f959d12-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2dd3ea32-301f-4c92-abce-5c667f959d12" (UID: "2dd3ea32-301f-4c92-abce-5c667f959d12"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:43:11.832232 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:11.832201    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dd3ea32-301f-4c92-abce-5c667f959d12-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:12.596919 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:12.596890    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf"
Jul 28 22:43:12.596919 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:12.596904    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf" event={"ID":"2dd3ea32-301f-4c92-abce-5c667f959d12","Type":"ContainerDied","Data":"3da81fb394487aa3e7fe7fc6a606b47f99ec12e874af5ff89a370750c3d102ca"}
Jul 28 22:43:12.597130 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:12.596951    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3da81fb394487aa3e7fe7fc6a606b47f99ec12e874af5ff89a370750c3d102ca"
Jul 28 22:43:16.705569 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:16.705536    2583 scope.go:117] "RemoveContainer" containerID="c6aaf8393c2a24df22789e5120f7620e3e8c33fb0b8bad6718eecb8827ab9c82"
Jul 28 22:43:19.995992 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:19.995958    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:20.167855 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:20.167790    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:20.251359 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:20.251269    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:20.295520 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:20.295483    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf"]
Jul 28 22:43:20.297684 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:20.297659    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a6fb9-xc4zf"]
Jul 28 22:43:20.315961 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:20.315931    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:20.431564 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:20.431530    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dd3ea32-301f-4c92-abce-5c667f959d12" path="/var/lib/kubelet/pods/2dd3ea32-301f-4c92-abce-5c667f959d12/volumes"
Jul 28 22:43:20.694141 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:20.694104    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:26.347478 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:26.347435    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:26.410530 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:26.410501    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:26.623071 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:26.623041    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"]
Jul 28 22:43:26.623295 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:26.623256    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c" podUID="331b742f-314f-4c56-8679-0f5d59181a2d" containerName="istio-proxy" containerID="cri-o://0213e846559514ed7a8265c074e3fd3e0af97b3961e06e02bd3ad529c6e8c420" gracePeriod=30
Jul 28 22:43:26.636868 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:26.636844    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:26.882999 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:26.882921    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:27.219895 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:27.219783    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:27.526541 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:27.526451    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:28.140832 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:28.140762    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:30.294146 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:30.294110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:30.379948 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:30.379916    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:30.441511 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:30.441482    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:30.541318 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:30.541288    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:30.961557 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:30.961523    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:31.061211 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:31.061181    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:31.881126 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:31.881104    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:43:32.003909 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.003830    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/331b742f-314f-4c56-8679-0f5d59181a2d-istio-podinfo\") pod \"331b742f-314f-4c56-8679-0f5d59181a2d\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") "
Jul 28 22:43:32.003909 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.003865    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-credential-socket\") pod \"331b742f-314f-4c56-8679-0f5d59181a2d\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") "
Jul 28 22:43:32.003909 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.003898    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/331b742f-314f-4c56-8679-0f5d59181a2d-istio-token\") pod \"331b742f-314f-4c56-8679-0f5d59181a2d\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") "
Jul 28 22:43:32.004184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.003926    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-istio-envoy\") pod \"331b742f-314f-4c56-8679-0f5d59181a2d\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") "
Jul 28 22:43:32.004184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.003940    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-istio-data\") pod \"331b742f-314f-4c56-8679-0f5d59181a2d\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") "
Jul 28 22:43:32.004184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.003984    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qg2tb\" (UniqueName: \"kubernetes.io/projected/331b742f-314f-4c56-8679-0f5d59181a2d-kube-api-access-qg2tb\") pod \"331b742f-314f-4c56-8679-0f5d59181a2d\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") "
Jul 28 22:43:32.004184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.004014    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-workload-socket\") pod \"331b742f-314f-4c56-8679-0f5d59181a2d\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") "
Jul 28 22:43:32.004184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.004053    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-workload-certs\") pod \"331b742f-314f-4c56-8679-0f5d59181a2d\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") "
Jul 28 22:43:32.004184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.004113    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/331b742f-314f-4c56-8679-0f5d59181a2d-istiod-ca-cert\") pod \"331b742f-314f-4c56-8679-0f5d59181a2d\" (UID: \"331b742f-314f-4c56-8679-0f5d59181a2d\") "
Jul 28 22:43:32.004485 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.004258    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "331b742f-314f-4c56-8679-0f5d59181a2d" (UID: "331b742f-314f-4c56-8679-0f5d59181a2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:32.004485 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.004420    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-credential-socket\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:32.004634 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.004605    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "331b742f-314f-4c56-8679-0f5d59181a2d" (UID: "331b742f-314f-4c56-8679-0f5d59181a2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:32.004701 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.004638    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "331b742f-314f-4c56-8679-0f5d59181a2d" (UID: "331b742f-314f-4c56-8679-0f5d59181a2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:32.004860 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.004839    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/331b742f-314f-4c56-8679-0f5d59181a2d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "331b742f-314f-4c56-8679-0f5d59181a2d" (UID: "331b742f-314f-4c56-8679-0f5d59181a2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:43:32.004913 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.004865    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-istio-data" (OuterVolumeSpecName: "istio-data") pod "331b742f-314f-4c56-8679-0f5d59181a2d" (UID: "331b742f-314f-4c56-8679-0f5d59181a2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:32.006263 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.006237    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/331b742f-314f-4c56-8679-0f5d59181a2d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "331b742f-314f-4c56-8679-0f5d59181a2d" (UID: "331b742f-314f-4c56-8679-0f5d59181a2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:43:32.006361 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.006274    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/331b742f-314f-4c56-8679-0f5d59181a2d-istio-token" (OuterVolumeSpecName: "istio-token") pod "331b742f-314f-4c56-8679-0f5d59181a2d" (UID: "331b742f-314f-4c56-8679-0f5d59181a2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:43:32.006403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.006389    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/331b742f-314f-4c56-8679-0f5d59181a2d-kube-api-access-qg2tb" (OuterVolumeSpecName: "kube-api-access-qg2tb") pod "331b742f-314f-4c56-8679-0f5d59181a2d" (UID: "331b742f-314f-4c56-8679-0f5d59181a2d"). InnerVolumeSpecName "kube-api-access-qg2tb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:43:32.006519 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.006500    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "331b742f-314f-4c56-8679-0f5d59181a2d" (UID: "331b742f-314f-4c56-8679-0f5d59181a2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:43:32.105543 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.105503    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/331b742f-314f-4c56-8679-0f5d59181a2d-istio-token\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:32.105543 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.105538    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-istio-envoy\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:32.105543 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.105547    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-istio-data\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:32.105543 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.105556    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qg2tb\" (UniqueName: \"kubernetes.io/projected/331b742f-314f-4c56-8679-0f5d59181a2d-kube-api-access-qg2tb\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:32.105774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.105566    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-workload-socket\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:32.105774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.105574    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/331b742f-314f-4c56-8679-0f5d59181a2d-workload-certs\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:32.105774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.105582    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/331b742f-314f-4c56-8679-0f5d59181a2d-istiod-ca-cert\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:32.105774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.105589    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/331b742f-314f-4c56-8679-0f5d59181a2d-istio-podinfo\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:43:32.702140 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.702054    2583 generic.go:356] "Generic (PLEG): container finished" podID="331b742f-314f-4c56-8679-0f5d59181a2d" containerID="0213e846559514ed7a8265c074e3fd3e0af97b3961e06e02bd3ad529c6e8c420" exitCode=0
Jul 28 22:43:32.702140 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.702099    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c" event={"ID":"331b742f-314f-4c56-8679-0f5d59181a2d","Type":"ContainerDied","Data":"0213e846559514ed7a8265c074e3fd3e0af97b3961e06e02bd3ad529c6e8c420"}
Jul 28 22:43:32.702140 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.702105    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"
Jul 28 22:43:32.702140 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.702140    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c" event={"ID":"331b742f-314f-4c56-8679-0f5d59181a2d","Type":"ContainerDied","Data":"3c368ead96b253eaefd5253466af25a5c5adfa54c939e4fdac1c6e9966402159"}
Jul 28 22:43:32.702404 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.702162    2583 scope.go:117] "RemoveContainer" containerID="0213e846559514ed7a8265c074e3fd3e0af97b3961e06e02bd3ad529c6e8c420"
Jul 28 22:43:32.711511 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.711490    2583 scope.go:117] "RemoveContainer" containerID="4fbbe935503ae3ff4216e61714492fd1afef019035a7e2d60988633dc411f830"
Jul 28 22:43:32.717621 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.717598    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"]
Jul 28 22:43:32.720192 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.720176    2583 scope.go:117] "RemoveContainer" containerID="0213e846559514ed7a8265c074e3fd3e0af97b3961e06e02bd3ad529c6e8c420"
Jul 28 22:43:32.720463 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:43:32.720446    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0213e846559514ed7a8265c074e3fd3e0af97b3961e06e02bd3ad529c6e8c420\": container with ID starting with 0213e846559514ed7a8265c074e3fd3e0af97b3961e06e02bd3ad529c6e8c420 not found: ID does not exist" containerID="0213e846559514ed7a8265c074e3fd3e0af97b3961e06e02bd3ad529c6e8c420"
Jul 28 22:43:32.720529 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.720469    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0213e846559514ed7a8265c074e3fd3e0af97b3961e06e02bd3ad529c6e8c420"} err="failed to get container status \"0213e846559514ed7a8265c074e3fd3e0af97b3961e06e02bd3ad529c6e8c420\": rpc error: code = NotFound desc = could not find container \"0213e846559514ed7a8265c074e3fd3e0af97b3961e06e02bd3ad529c6e8c420\": container with ID starting with 0213e846559514ed7a8265c074e3fd3e0af97b3961e06e02bd3ad529c6e8c420 not found: ID does not exist"
Jul 28 22:43:32.720529 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.720484    2583 scope.go:117] "RemoveContainer" containerID="4fbbe935503ae3ff4216e61714492fd1afef019035a7e2d60988633dc411f830"
Jul 28 22:43:32.720713 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:43:32.720698    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fbbe935503ae3ff4216e61714492fd1afef019035a7e2d60988633dc411f830\": container with ID starting with 4fbbe935503ae3ff4216e61714492fd1afef019035a7e2d60988633dc411f830 not found: ID does not exist" containerID="4fbbe935503ae3ff4216e61714492fd1afef019035a7e2d60988633dc411f830"
Jul 28 22:43:32.720775 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.720717    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fbbe935503ae3ff4216e61714492fd1afef019035a7e2d60988633dc411f830"} err="failed to get container status \"4fbbe935503ae3ff4216e61714492fd1afef019035a7e2d60988633dc411f830\": rpc error: code = NotFound desc = could not find container \"4fbbe935503ae3ff4216e61714492fd1afef019035a7e2d60988633dc411f830\": container with ID starting with 4fbbe935503ae3ff4216e61714492fd1afef019035a7e2d60988633dc411f830 not found: ID does not exist"
Jul 28 22:43:32.721704 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:32.721681    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-8a6fb9-openshift-default-69bc767cd4-hb82c"]
Jul 28 22:43:34.433236 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:34.433200    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="331b742f-314f-4c56-8679-0f5d59181a2d" path="/var/lib/kubelet/pods/331b742f-314f-4c56-8679-0f5d59181a2d/volumes"
Jul 28 22:43:55.385771 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:55.385738    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:56.027650 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.027615    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:56.225091 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.225058    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"]
Jul 28 22:43:56.225417 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.225404    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="331b742f-314f-4c56-8679-0f5d59181a2d" containerName="istio-proxy"
Jul 28 22:43:56.225417 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.225417    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="331b742f-314f-4c56-8679-0f5d59181a2d" containerName="istio-proxy"
Jul 28 22:43:56.225527 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.225426    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd3ea32-301f-4c92-abce-5c667f959d12" containerName="revoke-keys"
Jul 28 22:43:56.225527 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.225432    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd3ea32-301f-4c92-abce-5c667f959d12" containerName="revoke-keys"
Jul 28 22:43:56.225527 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.225441    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="331b742f-314f-4c56-8679-0f5d59181a2d" containerName="istio-proxy"
Jul 28 22:43:56.225527 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.225447    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="331b742f-314f-4c56-8679-0f5d59181a2d" containerName="istio-proxy"
Jul 28 22:43:56.225527 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.225501    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="331b742f-314f-4c56-8679-0f5d59181a2d" containerName="istio-proxy"
Jul 28 22:43:56.225527 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.225511    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dd3ea32-301f-4c92-abce-5c667f959d12" containerName="revoke-keys"
Jul 28 22:43:56.228535 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.228519    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.230425 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.230404    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-57f127-openshift-default-dockercfg-gvrlt\""
Jul 28 22:43:56.230540 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.230410    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 22:43:56.239270 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.239246    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"]
Jul 28 22:43:56.254181 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.254146    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:56.407585 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.407556    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-envoy\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.407981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.407588    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-credential-socket\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.407981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.407606    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-podinfo\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.407981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.407672    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-token\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.407981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.407690    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9kq8\" (UniqueName: \"kubernetes.io/projected/4ddca253-3f66-4507-a4f9-2ef9f252b73d-kube-api-access-n9kq8\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.407981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.407737    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-workload-socket\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.407981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.407756    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-workload-certs\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.407981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.407800    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-data\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.407981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.407839    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istiod-ca-cert\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.508450 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.508415    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-workload-socket\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.508450 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.508452    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-workload-certs\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.508706 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.508472    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-data\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.508706 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.508492    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istiod-ca-cert\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.508706 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.508534    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-envoy\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.508706 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.508554    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-credential-socket\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.508706 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.508569    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-podinfo\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.509009 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.508732    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-token\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.509009 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.508767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9kq8\" (UniqueName: \"kubernetes.io/projected/4ddca253-3f66-4507-a4f9-2ef9f252b73d-kube-api-access-n9kq8\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.509009 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.508931    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-workload-socket\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.509169 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.509011    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-workload-certs\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.509169 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.509025    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-data\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.509263 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.509189    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-credential-socket\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.509547 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.509527    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istiod-ca-cert\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.511160 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.511138    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-podinfo\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.511243 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.511146    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-envoy\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.518981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.518958    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-token\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.521074 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.521050    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9kq8\" (UniqueName: \"kubernetes.io/projected/4ddca253-3f66-4507-a4f9-2ef9f252b73d-kube-api-access-n9kq8\") pod \"e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") " pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.539976 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.539956    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:56.682933 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.682901    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"]
Jul 28 22:43:56.683418 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:43:56.683397    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ddca253_3f66_4507_a4f9_2ef9f252b73d.slice/crio-33ebb03cb228f72a7e88ed1848d063cdf0c81a2674f4eb52d75fdbae9034ee87 WatchSource:0}: Error finding container 33ebb03cb228f72a7e88ed1848d063cdf0c81a2674f4eb52d75fdbae9034ee87: Status 404 returned error can't find the container with id 33ebb03cb228f72a7e88ed1848d063cdf0c81a2674f4eb52d75fdbae9034ee87
Jul 28 22:43:56.685491 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.685465    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:43:56.685552 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.685525    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:43:56.685589 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.685554    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:43:56.839554 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.839523    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b" event={"ID":"4ddca253-3f66-4507-a4f9-2ef9f252b73d","Type":"ContainerStarted","Data":"99a7099e5490e7a0b864c77fbba9d341d27f2004b805da6a47fc4a2793b6f723"}
Jul 28 22:43:56.839713 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.839576    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b" event={"ID":"4ddca253-3f66-4507-a4f9-2ef9f252b73d","Type":"ContainerStarted","Data":"33ebb03cb228f72a7e88ed1848d063cdf0c81a2674f4eb52d75fdbae9034ee87"}
Jul 28 22:43:56.861325 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:56.861268    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b" podStartSLOduration=0.861247635 podStartE2EDuration="861.247635ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:43:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:43:56.857520485 +0000 UTC m=+2380.865386046" watchObservedRunningTime="2026-07-28 22:43:56.861247635 +0000 UTC m=+2380.869113195"
Jul 28 22:43:57.019976 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:57.019899    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:57.540614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:57.540579    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:57.545254 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:57.545231    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:57.845530 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:57.845439    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:57.846281 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:57.846260    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:43:58.824301 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:58.824255    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:58.924853 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:58.924818    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:59.027987 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:59.027957    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:59.144263 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:59.144230    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:59.530470 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:59.530382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:43:59.624610 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:43:59.624579    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:44:15.949155 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:15.949124    2583 generic.go:356] "Generic (PLEG): container finished" podID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" containerID="99a7099e5490e7a0b864c77fbba9d341d27f2004b805da6a47fc4a2793b6f723" exitCode=0
Jul 28 22:44:15.949528 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:15.949198    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b" event={"ID":"4ddca253-3f66-4507-a4f9-2ef9f252b73d","Type":"ContainerDied","Data":"99a7099e5490e7a0b864c77fbba9d341d27f2004b805da6a47fc4a2793b6f723"}
Jul 28 22:44:15.949609 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:15.949596    2583 scope.go:117] "RemoveContainer" containerID="99a7099e5490e7a0b864c77fbba9d341d27f2004b805da6a47fc4a2793b6f723"
Jul 28 22:44:15.950580 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:15.950556    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:44:15.950659 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:15.950620    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:44:15.950659 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:15.950653    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:44:16.585765 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:16.585739    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:44:16.604110 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:16.604089    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:44:16.960999 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:16.960953    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b" event={"ID":"4ddca253-3f66-4507-a4f9-2ef9f252b73d","Type":"ContainerStarted","Data":"3e0a72a839db498bb1b7737d12ff549e44725f0475b83113e5679cc6bfe79cc0"}
Jul 28 22:44:17.541163 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:17.541124    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:44:17.541367 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:17.541289    2583 patch_prober.go:28] interesting pod/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 28 22:44:17.541367 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:17.541334    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b" podUID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 28 22:44:18.541069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:18.541025    2583 patch_prober.go:28] interesting pod/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 28 22:44:18.541555 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:18.541092    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b" podUID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 28 22:44:19.540564 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:19.540516    2583 patch_prober.go:28] interesting pod/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 28 22:44:19.540776 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:19.540591    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b" podUID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 28 22:44:20.541007 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:20.540968    2583 patch_prober.go:28] interesting pod/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 28 22:44:20.541577 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:20.541546    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b" podUID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 28 22:44:21.544291 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:21.544260    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:44:21.544711 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:21.544516    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:44:21.545137 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:21.545119    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:44:23.978242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:23.978207    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:44:27.260250 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:27.260215    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx"]
Jul 28 22:44:27.260718 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:27.260626    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="331b742f-314f-4c56-8679-0f5d59181a2d" containerName="istio-proxy"
Jul 28 22:44:27.263575 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:27.263560    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx"
Jul 28 22:44:27.270050 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:27.270021    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:44:27.276024 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:27.276000    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx"]
Jul 28 22:44:27.375921 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:27.375890    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acc0d66-503a-406b-b5d4-23867d6a9f74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx\" (UID: \"6acc0d66-503a-406b-b5d4-23867d6a9f74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx"
Jul 28 22:44:27.477519 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:27.477419    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acc0d66-503a-406b-b5d4-23867d6a9f74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx\" (UID: \"6acc0d66-503a-406b-b5d4-23867d6a9f74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx"
Jul 28 22:44:27.478022 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:27.478003    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acc0d66-503a-406b-b5d4-23867d6a9f74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx\" (UID: \"6acc0d66-503a-406b-b5d4-23867d6a9f74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx"
Jul 28 22:44:27.574357 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:27.574273    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx"
Jul 28 22:44:27.722702 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:27.722679    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx"]
Jul 28 22:44:27.724421 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:44:27.724393    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6acc0d66_503a_406b_b5d4_23867d6a9f74.slice/crio-b9d5c45f8b1b577fb9ebb013c88787044a4defea9acdbbcb5b500dac1e3a3413 WatchSource:0}: Error finding container b9d5c45f8b1b577fb9ebb013c88787044a4defea9acdbbcb5b500dac1e3a3413: Status 404 returned error can't find the container with id b9d5c45f8b1b577fb9ebb013c88787044a4defea9acdbbcb5b500dac1e3a3413
Jul 28 22:44:28.020035 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:28.019999    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx" event={"ID":"6acc0d66-503a-406b-b5d4-23867d6a9f74","Type":"ContainerStarted","Data":"b9d5c45f8b1b577fb9ebb013c88787044a4defea9acdbbcb5b500dac1e3a3413"}
Jul 28 22:44:29.026458 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:29.026423    2583 generic.go:356] "Generic (PLEG): container finished" podID="6acc0d66-503a-406b-b5d4-23867d6a9f74" containerID="1e266fde6bcf69ebe5430fd8f764872b22d5a503bdaa3ecb958748c2a1389307" exitCode=0
Jul 28 22:44:29.026893 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:29.026513    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx" event={"ID":"6acc0d66-503a-406b-b5d4-23867d6a9f74","Type":"ContainerDied","Data":"1e266fde6bcf69ebe5430fd8f764872b22d5a503bdaa3ecb958748c2a1389307"}
Jul 28 22:44:30.164258 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:30.164232    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx"
Jul 28 22:44:30.302127 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:30.302040    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acc0d66-503a-406b-b5d4-23867d6a9f74-maas-api-service-ca\") pod \"6acc0d66-503a-406b-b5d4-23867d6a9f74\" (UID: \"6acc0d66-503a-406b-b5d4-23867d6a9f74\") "
Jul 28 22:44:30.302448 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:30.302420    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6acc0d66-503a-406b-b5d4-23867d6a9f74-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6acc0d66-503a-406b-b5d4-23867d6a9f74" (UID: "6acc0d66-503a-406b-b5d4-23867d6a9f74"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:44:30.402962 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:30.402933    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6acc0d66-503a-406b-b5d4-23867d6a9f74-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:44:31.039992 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:31.039962    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx"
Jul 28 22:44:31.040180 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:31.040007    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx" event={"ID":"6acc0d66-503a-406b-b5d4-23867d6a9f74","Type":"ContainerDied","Data":"b9d5c45f8b1b577fb9ebb013c88787044a4defea9acdbbcb5b500dac1e3a3413"}
Jul 28 22:44:31.040180 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:31.040064    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9d5c45f8b1b577fb9ebb013c88787044a4defea9acdbbcb5b500dac1e3a3413"
Jul 28 22:44:42.851989 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:42.851895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:44:42.953040 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:42.953003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:44:43.005361 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:43.005330    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx"]
Jul 28 22:44:43.007627 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:43.007604    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eeffd6-2m6lx"]
Jul 28 22:44:43.013733 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:43.013703    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:44:43.409919 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:43.409889    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:44:44.431128 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:44.431094    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6acc0d66-503a-406b-b5d4-23867d6a9f74" path="/var/lib/kubelet/pods/6acc0d66-503a-406b-b5d4-23867d6a9f74/volumes"
Jul 28 22:44:49.182392 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:49.182336    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:44:49.236598 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:49.236572    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:44:49.467041 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:49.466949    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:44:49.701737 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:49.701700    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:44:49.828927 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:49.828843    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft"]
Jul 28 22:44:49.829244 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:49.829231    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6acc0d66-503a-406b-b5d4-23867d6a9f74" containerName="revoke-keys"
Jul 28 22:44:49.829299 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:49.829246    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6acc0d66-503a-406b-b5d4-23867d6a9f74" containerName="revoke-keys"
Jul 28 22:44:49.829337 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:49.829306    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6acc0d66-503a-406b-b5d4-23867d6a9f74" containerName="revoke-keys"
Jul 28 22:44:49.832501 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:49.832486    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft"
Jul 28 22:44:49.835346 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:49.835317    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:44:49.837860 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:49.837832    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft"]
Jul 28 22:44:49.975378 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:49.975344    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4942ddd0-030f-4f75-a9a4-9c2744b5bd0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-57f127-nrzft\" (UID: \"4942ddd0-030f-4f75-a9a4-9c2744b5bd0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft"
Jul 28 22:44:50.075868 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:50.075827    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4942ddd0-030f-4f75-a9a4-9c2744b5bd0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-57f127-nrzft\" (UID: \"4942ddd0-030f-4f75-a9a4-9c2744b5bd0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft"
Jul 28 22:44:50.076420 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:50.076402    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4942ddd0-030f-4f75-a9a4-9c2744b5bd0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-57f127-nrzft\" (UID: \"4942ddd0-030f-4f75-a9a4-9c2744b5bd0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft"
Jul 28 22:44:50.145940 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:50.145912    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft"
Jul 28 22:44:50.282371 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:50.282337    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft"]
Jul 28 22:44:50.284192 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:44:50.284168    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4942ddd0_030f_4f75_a9a4_9c2744b5bd0b.slice/crio-926a541a05fef9a630d68f1a7f442e7aacf781c33a140301abce355a68f2f46b WatchSource:0}: Error finding container 926a541a05fef9a630d68f1a7f442e7aacf781c33a140301abce355a68f2f46b: Status 404 returned error can't find the container with id 926a541a05fef9a630d68f1a7f442e7aacf781c33a140301abce355a68f2f46b
Jul 28 22:44:51.160343 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:51.160311    2583 generic.go:356] "Generic (PLEG): container finished" podID="4942ddd0-030f-4f75-a9a4-9c2744b5bd0b" containerID="55acb90cea2b212ca11217919bd8b2c9ed6c0b539ad89f17aeb7a1d957161abf" exitCode=0
Jul 28 22:44:51.160535 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:51.160392    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft" event={"ID":"4942ddd0-030f-4f75-a9a4-9c2744b5bd0b","Type":"ContainerDied","Data":"55acb90cea2b212ca11217919bd8b2c9ed6c0b539ad89f17aeb7a1d957161abf"}
Jul 28 22:44:51.160535 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:51.160443    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft" event={"ID":"4942ddd0-030f-4f75-a9a4-9c2744b5bd0b","Type":"ContainerStarted","Data":"926a541a05fef9a630d68f1a7f442e7aacf781c33a140301abce355a68f2f46b"}
Jul 28 22:44:52.295588 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:52.295562    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft"
Jul 28 22:44:52.397994 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:52.397941    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4942ddd0-030f-4f75-a9a4-9c2744b5bd0b-maas-api-service-ca\") pod \"4942ddd0-030f-4f75-a9a4-9c2744b5bd0b\" (UID: \"4942ddd0-030f-4f75-a9a4-9c2744b5bd0b\") "
Jul 28 22:44:52.398324 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:52.398302    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4942ddd0-030f-4f75-a9a4-9c2744b5bd0b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4942ddd0-030f-4f75-a9a4-9c2744b5bd0b" (UID: "4942ddd0-030f-4f75-a9a4-9c2744b5bd0b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:44:52.499539 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:52.499501    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4942ddd0-030f-4f75-a9a4-9c2744b5bd0b-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:44:53.173395 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:53.173367    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft"
Jul 28 22:44:53.173570 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:53.173414    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft" event={"ID":"4942ddd0-030f-4f75-a9a4-9c2744b5bd0b","Type":"ContainerDied","Data":"926a541a05fef9a630d68f1a7f442e7aacf781c33a140301abce355a68f2f46b"}
Jul 28 22:44:53.173570 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:44:53.173464    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="926a541a05fef9a630d68f1a7f442e7aacf781c33a140301abce355a68f2f46b"
Jul 28 22:45:00.141975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.141929    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754645-pk5zr"]
Jul 28 22:45:00.142507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.142480    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4942ddd0-030f-4f75-a9a4-9c2744b5bd0b" containerName="revoke-keys"
Jul 28 22:45:00.142507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.142506    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4942ddd0-030f-4f75-a9a4-9c2744b5bd0b" containerName="revoke-keys"
Jul 28 22:45:00.142672 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.142589    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4942ddd0-030f-4f75-a9a4-9c2744b5bd0b" containerName="revoke-keys"
Jul 28 22:45:00.145595 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.145579    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj"]
Jul 28 22:45:00.145718 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.145696    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754645-pk5zr"
Jul 28 22:45:00.147959 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.147935    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:45:00.148690 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.148676    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj"
Jul 28 22:45:00.155723 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.154427    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754645-pk5zr"]
Jul 28 22:45:00.157686 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.157662    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj"]
Jul 28 22:45:00.161234 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.161204    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754645-pk5zr"
Jul 28 22:45:00.167574 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.167555    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj"
Jul 28 22:45:00.322829 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.322774    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754645-pk5zr"]
Jul 28 22:45:00.324947 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:45:00.324918    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7460df6e_7d95_4c3d_999c_3191081cf666.slice/crio-c3f197aa275ce4395075965e612bb60f18a135a23e1d675b8b026c0e4a73aee2 WatchSource:0}: Error finding container c3f197aa275ce4395075965e612bb60f18a135a23e1d675b8b026c0e4a73aee2: Status 404 returned error can't find the container with id c3f197aa275ce4395075965e612bb60f18a135a23e1d675b8b026c0e4a73aee2
Jul 28 22:45:00.348008 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:00.347983    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj"]
Jul 28 22:45:00.348695 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:45:00.348668    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd344f718_38d4_49b3_83c4_b2fccb48c766.slice/crio-f2b05068f8a99c0e09e5062ff12dbb1b2c89d0de23bde349669ae0e0e95e608d WatchSource:0}: Error finding container f2b05068f8a99c0e09e5062ff12dbb1b2c89d0de23bde349669ae0e0e95e608d: Status 404 returned error can't find the container with id f2b05068f8a99c0e09e5062ff12dbb1b2c89d0de23bde349669ae0e0e95e608d
Jul 28 22:45:01.228714 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:01.228675    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj" event={"ID":"d344f718-38d4-49b3-83c4-b2fccb48c766","Type":"ContainerStarted","Data":"5bd810ffccd5422935270d074971dbcbc054fb65db7e3f209f47620bb7d4577c"}
Jul 28 22:45:01.229211 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:01.228770    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj" event={"ID":"d344f718-38d4-49b3-83c4-b2fccb48c766","Type":"ContainerStarted","Data":"f2b05068f8a99c0e09e5062ff12dbb1b2c89d0de23bde349669ae0e0e95e608d"}
Jul 28 22:45:01.230409 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:01.230383    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754645-pk5zr" event={"ID":"7460df6e-7d95-4c3d-999c-3191081cf666","Type":"ContainerStarted","Data":"af81e4f6b60bc8691a97a83cce5ce852c27703272039d7d9f1e7e2892fff9a1d"}
Jul 28 22:45:01.230547 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:01.230421    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754645-pk5zr" event={"ID":"7460df6e-7d95-4c3d-999c-3191081cf666","Type":"ContainerStarted","Data":"c3f197aa275ce4395075965e612bb60f18a135a23e1d675b8b026c0e4a73aee2"}
Jul 28 22:45:01.243292 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:01.243250    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj" podStartSLOduration=1.243238454 podStartE2EDuration="1.243238454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:45:01.241363262 +0000 UTC m=+2445.249228844" watchObservedRunningTime="2026-07-28 22:45:01.243238454 +0000 UTC m=+2445.251104040"
Jul 28 22:45:01.256058 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:01.256020    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754645-pk5zr" podStartSLOduration=1.256008852 podStartE2EDuration="1.256008852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:45:01.253467588 +0000 UTC m=+2445.261333169" watchObservedRunningTime="2026-07-28 22:45:01.256008852 +0000 UTC m=+2445.263874409"
Jul 28 22:45:02.237352 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:02.237318    2583 generic.go:356] "Generic (PLEG): container finished" podID="d344f718-38d4-49b3-83c4-b2fccb48c766" containerID="5bd810ffccd5422935270d074971dbcbc054fb65db7e3f209f47620bb7d4577c" exitCode=0
Jul 28 22:45:02.237870 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:02.237394    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj" event={"ID":"d344f718-38d4-49b3-83c4-b2fccb48c766","Type":"ContainerDied","Data":"5bd810ffccd5422935270d074971dbcbc054fb65db7e3f209f47620bb7d4577c"}
Jul 28 22:45:02.238820 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:02.238788    2583 generic.go:356] "Generic (PLEG): container finished" podID="7460df6e-7d95-4c3d-999c-3191081cf666" containerID="af81e4f6b60bc8691a97a83cce5ce852c27703272039d7d9f1e7e2892fff9a1d" exitCode=0
Jul 28 22:45:02.238917 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:02.238825    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754645-pk5zr" event={"ID":"7460df6e-7d95-4c3d-999c-3191081cf666","Type":"ContainerDied","Data":"af81e4f6b60bc8691a97a83cce5ce852c27703272039d7d9f1e7e2892fff9a1d"}
Jul 28 22:45:03.372434 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:03.372411    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754645-pk5zr"
Jul 28 22:45:03.407082 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:03.407058    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj"
Jul 28 22:45:04.251583 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:04.251551    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj"
Jul 28 22:45:04.251583 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:04.251559    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj" event={"ID":"d344f718-38d4-49b3-83c4-b2fccb48c766","Type":"ContainerDied","Data":"f2b05068f8a99c0e09e5062ff12dbb1b2c89d0de23bde349669ae0e0e95e608d"}
Jul 28 22:45:04.251857 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:04.251604    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2b05068f8a99c0e09e5062ff12dbb1b2c89d0de23bde349669ae0e0e95e608d"
Jul 28 22:45:04.253121 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:04.253099    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754645-pk5zr"
Jul 28 22:45:04.253121 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:04.253105    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754645-pk5zr" event={"ID":"7460df6e-7d95-4c3d-999c-3191081cf666","Type":"ContainerDied","Data":"c3f197aa275ce4395075965e612bb60f18a135a23e1d675b8b026c0e4a73aee2"}
Jul 28 22:45:04.253263 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:04.253141    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3f197aa275ce4395075965e612bb60f18a135a23e1d675b8b026c0e4a73aee2"
Jul 28 22:45:05.288873 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:05.288834    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:05.289401 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:05.289381    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj"]
Jul 28 22:45:05.294100 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:05.294076    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-57f127-29754645-cbmkj"]
Jul 28 22:45:05.359649 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:05.359612    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:05.424280 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:05.424242    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:05.451647 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:05.451617    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft"]
Jul 28 22:45:05.453718 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:05.453697    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-57f127-nrzft"]
Jul 28 22:45:05.488553 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:05.488515    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:05.879352 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:05.879318    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:06.432006 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:06.431972    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4942ddd0-030f-4f75-a9a4-9c2744b5bd0b" path="/var/lib/kubelet/pods/4942ddd0-030f-4f75-a9a4-9c2744b5bd0b/volumes"
Jul 28 22:45:06.432378 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:06.432298    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d344f718-38d4-49b3-83c4-b2fccb48c766" path="/var/lib/kubelet/pods/d344f718-38d4-49b3-83c4-b2fccb48c766/volumes"
Jul 28 22:45:12.015152 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:12.015110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:12.115172 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:12.115138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:12.292055 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:12.291978    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"]
Jul 28 22:45:12.292277 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:12.292252    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b" podUID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" containerName="istio-proxy" containerID="cri-o://3e0a72a839db498bb1b7737d12ff549e44725f0475b83113e5679cc6bfe79cc0" gracePeriod=30
Jul 28 22:45:12.332318 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:12.332289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:12.570157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:12.570076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:12.959571 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:12.959540    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:13.297975 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:13.297896    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:13.819549 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:13.819515    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:15.889320 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:15.889288    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:15.996774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:15.996709    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:16.103102 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:16.103068    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:16.322184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:16.322103    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:16.669247 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:16.669209    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:16.752505 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:16.752478    2583 scope.go:117] "RemoveContainer" containerID="e2a55870f209d63701f0fbeaf3e48d44579849ce14eda0f34d34f1d19140b915"
Jul 28 22:45:16.776257 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:16.776225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:17.547051 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.547028    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:45:17.595857 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.595753    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-envoy\") pod \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") "
Jul 28 22:45:17.595857 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.595782    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-credential-socket\") pod \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") "
Jul 28 22:45:17.596069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.595867    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-data\") pod \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") "
Jul 28 22:45:17.596069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.595992    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-podinfo\") pod \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") "
Jul 28 22:45:17.596069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.596026    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istiod-ca-cert\") pod \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") "
Jul 28 22:45:17.596069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.596053    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-workload-socket\") pod \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") "
Jul 28 22:45:17.596609 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.596079    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-workload-certs\") pod \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") "
Jul 28 22:45:17.596609 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.596123    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-token\") pod \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") "
Jul 28 22:45:17.596609 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.596152    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9kq8\" (UniqueName: \"kubernetes.io/projected/4ddca253-3f66-4507-a4f9-2ef9f252b73d-kube-api-access-n9kq8\") pod \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\" (UID: \"4ddca253-3f66-4507-a4f9-2ef9f252b73d\") "
Jul 28 22:45:17.596609 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.596051    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4ddca253-3f66-4507-a4f9-2ef9f252b73d" (UID: "4ddca253-3f66-4507-a4f9-2ef9f252b73d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:45:17.596609 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.596255    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4ddca253-3f66-4507-a4f9-2ef9f252b73d" (UID: "4ddca253-3f66-4507-a4f9-2ef9f252b73d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:45:17.596609 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.596446    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4ddca253-3f66-4507-a4f9-2ef9f252b73d" (UID: "4ddca253-3f66-4507-a4f9-2ef9f252b73d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:45:17.596609 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.596471    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-workload-socket\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:45:17.596609 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.596491    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-credential-socket\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:45:17.597081 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.596878    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4ddca253-3f66-4507-a4f9-2ef9f252b73d" (UID: "4ddca253-3f66-4507-a4f9-2ef9f252b73d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:45:17.597597 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.597571    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-data" (OuterVolumeSpecName: "istio-data") pod "4ddca253-3f66-4507-a4f9-2ef9f252b73d" (UID: "4ddca253-3f66-4507-a4f9-2ef9f252b73d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:45:17.598456 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.598430    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ddca253-3f66-4507-a4f9-2ef9f252b73d-kube-api-access-n9kq8" (OuterVolumeSpecName: "kube-api-access-n9kq8") pod "4ddca253-3f66-4507-a4f9-2ef9f252b73d" (UID: "4ddca253-3f66-4507-a4f9-2ef9f252b73d"). InnerVolumeSpecName "kube-api-access-n9kq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:45:17.598557 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.598452    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-token" (OuterVolumeSpecName: "istio-token") pod "4ddca253-3f66-4507-a4f9-2ef9f252b73d" (UID: "4ddca253-3f66-4507-a4f9-2ef9f252b73d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:45:17.598619 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.598602    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4ddca253-3f66-4507-a4f9-2ef9f252b73d" (UID: "4ddca253-3f66-4507-a4f9-2ef9f252b73d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:45:17.598619 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.598610    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4ddca253-3f66-4507-a4f9-2ef9f252b73d" (UID: "4ddca253-3f66-4507-a4f9-2ef9f252b73d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:45:17.697302 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.697279    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-podinfo\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:45:17.697302 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.697302    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istiod-ca-cert\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:45:17.697459 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.697311    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-workload-certs\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:45:17.697459 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.697320    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-token\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:45:17.697459 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.697329    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9kq8\" (UniqueName: \"kubernetes.io/projected/4ddca253-3f66-4507-a4f9-2ef9f252b73d-kube-api-access-n9kq8\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:45:17.697459 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.697337    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-envoy\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:45:17.697459 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:17.697345    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ddca253-3f66-4507-a4f9-2ef9f252b73d-istio-data\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:45:18.333755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:18.333722    2583 generic.go:356] "Generic (PLEG): container finished" podID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" containerID="3e0a72a839db498bb1b7737d12ff549e44725f0475b83113e5679cc6bfe79cc0" exitCode=0
Jul 28 22:45:18.333935 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:18.333771    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"
Jul 28 22:45:18.333935 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:18.333831    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b" event={"ID":"4ddca253-3f66-4507-a4f9-2ef9f252b73d","Type":"ContainerDied","Data":"3e0a72a839db498bb1b7737d12ff549e44725f0475b83113e5679cc6bfe79cc0"}
Jul 28 22:45:18.333935 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:18.333883    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b" event={"ID":"4ddca253-3f66-4507-a4f9-2ef9f252b73d","Type":"ContainerDied","Data":"33ebb03cb228f72a7e88ed1848d063cdf0c81a2674f4eb52d75fdbae9034ee87"}
Jul 28 22:45:18.333935 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:18.333908    2583 scope.go:117] "RemoveContainer" containerID="3e0a72a839db498bb1b7737d12ff549e44725f0475b83113e5679cc6bfe79cc0"
Jul 28 22:45:18.343434 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:18.343417    2583 scope.go:117] "RemoveContainer" containerID="99a7099e5490e7a0b864c77fbba9d341d27f2004b805da6a47fc4a2793b6f723"
Jul 28 22:45:18.351896 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:18.351779    2583 scope.go:117] "RemoveContainer" containerID="3e0a72a839db498bb1b7737d12ff549e44725f0475b83113e5679cc6bfe79cc0"
Jul 28 22:45:18.352135 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:45:18.352108    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e0a72a839db498bb1b7737d12ff549e44725f0475b83113e5679cc6bfe79cc0\": container with ID starting with 3e0a72a839db498bb1b7737d12ff549e44725f0475b83113e5679cc6bfe79cc0 not found: ID does not exist" containerID="3e0a72a839db498bb1b7737d12ff549e44725f0475b83113e5679cc6bfe79cc0"
Jul 28 22:45:18.352243 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:18.352143    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e0a72a839db498bb1b7737d12ff549e44725f0475b83113e5679cc6bfe79cc0"} err="failed to get container status \"3e0a72a839db498bb1b7737d12ff549e44725f0475b83113e5679cc6bfe79cc0\": rpc error: code = NotFound desc = could not find container \"3e0a72a839db498bb1b7737d12ff549e44725f0475b83113e5679cc6bfe79cc0\": container with ID starting with 3e0a72a839db498bb1b7737d12ff549e44725f0475b83113e5679cc6bfe79cc0 not found: ID does not exist"
Jul 28 22:45:18.352243 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:18.352168    2583 scope.go:117] "RemoveContainer" containerID="99a7099e5490e7a0b864c77fbba9d341d27f2004b805da6a47fc4a2793b6f723"
Jul 28 22:45:18.352457 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:45:18.352438    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99a7099e5490e7a0b864c77fbba9d341d27f2004b805da6a47fc4a2793b6f723\": container with ID starting with 99a7099e5490e7a0b864c77fbba9d341d27f2004b805da6a47fc4a2793b6f723 not found: ID does not exist" containerID="99a7099e5490e7a0b864c77fbba9d341d27f2004b805da6a47fc4a2793b6f723"
Jul 28 22:45:18.352519 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:18.352463    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99a7099e5490e7a0b864c77fbba9d341d27f2004b805da6a47fc4a2793b6f723"} err="failed to get container status \"99a7099e5490e7a0b864c77fbba9d341d27f2004b805da6a47fc4a2793b6f723\": rpc error: code = NotFound desc = could not find container \"99a7099e5490e7a0b864c77fbba9d341d27f2004b805da6a47fc4a2793b6f723\": container with ID starting with 99a7099e5490e7a0b864c77fbba9d341d27f2004b805da6a47fc4a2793b6f723 not found: ID does not exist"
Jul 28 22:45:18.359871 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:18.356572    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"]
Jul 28 22:45:18.361638 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:18.361616    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-57f127-openshift-default-dbf49fdb6-h7r5b"]
Jul 28 22:45:18.432060 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:18.432034    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" path="/var/lib/kubelet/pods/4ddca253-3f66-4507-a4f9-2ef9f252b73d/volumes"
Jul 28 22:45:40.573438 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:40.573399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:40.636562 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:40.636522    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:41.000058 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.000009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:41.382209 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.382170    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:41.672612 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.672528    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"]
Jul 28 22:45:41.673045 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.673029    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" containerName="istio-proxy"
Jul 28 22:45:41.673089 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.673047    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" containerName="istio-proxy"
Jul 28 22:45:41.673089 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.673057    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7460df6e-7d95-4c3d-999c-3191081cf666" containerName="cleanup"
Jul 28 22:45:41.673089 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.673063    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7460df6e-7d95-4c3d-999c-3191081cf666" containerName="cleanup"
Jul 28 22:45:41.673089 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.673073    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" containerName="istio-proxy"
Jul 28 22:45:41.673089 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.673078    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" containerName="istio-proxy"
Jul 28 22:45:41.673248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.673096    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d344f718-38d4-49b3-83c4-b2fccb48c766" containerName="cleanup"
Jul 28 22:45:41.673248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.673102    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d344f718-38d4-49b3-83c4-b2fccb48c766" containerName="cleanup"
Jul 28 22:45:41.673248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.673158    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d344f718-38d4-49b3-83c4-b2fccb48c766" containerName="cleanup"
Jul 28 22:45:41.673248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.673169    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7460df6e-7d95-4c3d-999c-3191081cf666" containerName="cleanup"
Jul 28 22:45:41.673248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.673174    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" containerName="istio-proxy"
Jul 28 22:45:41.673248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.673181    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ddca253-3f66-4507-a4f9-2ef9f252b73d" containerName="istio-proxy"
Jul 28 22:45:41.676502 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.676482    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.678408 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.678387    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 22:45:41.678523 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.678483    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-1661d9-openshift-default-dockercfg-q7jsh\""
Jul 28 22:45:41.687289 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.687263    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"]
Jul 28 22:45:41.694604 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.694577    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:41.805539 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.805503    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-workload-socket\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.805539 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.805540    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-workload-certs\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.805765 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.805571    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istiod-ca-cert\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.805765 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.805619    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-envoy\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.805765 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.805649    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-podinfo\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.805765 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.805680    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-credential-socket\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.805765 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.805710    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-data\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.805765 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.805730    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-token\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.806025 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.805826    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9k2w\" (UniqueName: \"kubernetes.io/projected/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-kube-api-access-l9k2w\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.907010 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.906973    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-podinfo\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.907010 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.907010    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-credential-socket\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.907247 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.907037    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-data\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.907247 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.907129    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-token\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.907247 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.907199    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9k2w\" (UniqueName: \"kubernetes.io/projected/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-kube-api-access-l9k2w\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.907247 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.907225    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-workload-socket\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.907247 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.907241    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-workload-certs\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.907500 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.907262    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istiod-ca-cert\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.907500 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.907312    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-envoy\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.907500 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.907388    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-data\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.907500 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.907433    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-credential-socket\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.907716 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.907696    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-workload-socket\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.907789 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.907773    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-workload-certs\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.908092 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.908067    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istiod-ca-cert\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.909423 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.909396    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-envoy\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.909513 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.909500    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-podinfo\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.910522 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.910493    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:41.914071 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.914045    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9k2w\" (UniqueName: \"kubernetes.io/projected/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-kube-api-access-l9k2w\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.914187 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.914171    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-token\") pod \"e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") " pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:41.989176 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:41.989114    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:42.126365 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:42.126333    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"]
Jul 28 22:45:42.127465 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:45:42.127434    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7f8513_6bc2_4e6d_8ef0_e323b64b1fad.slice/crio-7d3a6f5c3ad45ce65fe326c9dddd557c134a332f8c5a5dc596ca20060a3b6595 WatchSource:0}: Error finding container 7d3a6f5c3ad45ce65fe326c9dddd557c134a332f8c5a5dc596ca20060a3b6595: Status 404 returned error can't find the container with id 7d3a6f5c3ad45ce65fe326c9dddd557c134a332f8c5a5dc596ca20060a3b6595
Jul 28 22:45:42.129547 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:42.129514    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:45:42.129637 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:42.129578    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:45:42.129637 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:42.129605    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:45:42.470721 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:42.470680    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" event={"ID":"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad","Type":"ContainerStarted","Data":"149ed76841a77673a189be4fa727f76a2c1510d6a17595a587d490cb7fcc89d4"}
Jul 28 22:45:42.470931 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:42.470751    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" event={"ID":"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad","Type":"ContainerStarted","Data":"7d3a6f5c3ad45ce65fe326c9dddd557c134a332f8c5a5dc596ca20060a3b6595"}
Jul 28 22:45:42.489438 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:42.489389    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" podStartSLOduration=1.489373403 podStartE2EDuration="1.489373403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:45:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:45:42.488095201 +0000 UTC m=+2486.495960759" watchObservedRunningTime="2026-07-28 22:45:42.489373403 +0000 UTC m=+2486.497238969"
Jul 28 22:45:42.989528 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:42.989493    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:43.994330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:43.994299    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:44.459385 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:44.459350    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:44.481235 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:44.481212    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:44.482231 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:44.482213    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:45:44.576074 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:44.576047    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:44.754681 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:44.754601    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:44.910881 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:44.910849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:45.241751 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:45.241715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:45.343429 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:45.343393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:53.709889 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:53.709854    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:45:53.766239 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:45:53.766202    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:46:00.203094 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:00.202348    2583 patch_prober.go:28] interesting pod/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": read tcp 10.132.0.2:47656->10.132.0.69:15021: read: connection reset by peer" start-of-body=
Jul 28 22:46:00.203094 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:00.202413    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": read tcp 10.132.0.2:47656->10.132.0.69:15021: read: connection reset by peer"
Jul 28 22:46:00.587675 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:00.587592    2583 generic.go:356] "Generic (PLEG): container finished" podID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerID="149ed76841a77673a189be4fa727f76a2c1510d6a17595a587d490cb7fcc89d4" exitCode=0
Jul 28 22:46:00.587675 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:00.587636    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" event={"ID":"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad","Type":"ContainerDied","Data":"149ed76841a77673a189be4fa727f76a2c1510d6a17595a587d490cb7fcc89d4"}
Jul 28 22:46:00.588020 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:00.588006    2583 scope.go:117] "RemoveContainer" containerID="149ed76841a77673a189be4fa727f76a2c1510d6a17595a587d490cb7fcc89d4"
Jul 28 22:46:00.589064 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:00.589037    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:46:00.589137 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:00.589121    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:46:00.589176 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:00.589164    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:46:01.595613 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:01.595579    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" event={"ID":"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad","Type":"ContainerStarted","Data":"e5ef94495e183bef4d90be8da387660c87d4aa34fae563b6b69e0be5ca12549b"}
Jul 28 22:46:01.989820 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:01.989757    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:46:01.990024 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:01.990003    2583 patch_prober.go:28] interesting pod/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Jul 28 22:46:01.990106 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:01.990060    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Jul 28 22:46:02.989538 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:02.989489    2583 patch_prober.go:28] interesting pod/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Jul 28 22:46:02.990032 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:02.989554    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Jul 28 22:46:03.989504 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:03.989462    2583 patch_prober.go:28] interesting pod/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Jul 28 22:46:03.989697 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:03.989524    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Jul 28 22:46:04.990116 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:04.990077    2583 patch_prober.go:28] interesting pod/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Jul 28 22:46:04.990504 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:04.990150    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Jul 28 22:46:05.992931 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:05.992899    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:46:05.993317 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:05.993268    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:46:05.993948 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:05.993931    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:46:09.534778 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:09.534699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:46:17.280053 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:17.280009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:46:36.723036 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:46:36.722986    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:48:12.749686 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:48:12.749656    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:49:16.620540 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:49:16.620468    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:49:16.644178 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:49:16.644151    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:49:16.804520 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:49:16.804486    2583 scope.go:117] "RemoveContainer" containerID="f779c90d0b1cdbf9911867db59bc9d2d4301a44b721dc6127cc122462f6367a8"
Jul 28 22:49:16.813139 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:49:16.813116    2583 scope.go:117] "RemoveContainer" containerID="e5d909733603c728eaa78c5c5ffcbbda52dff2d2d58a90b11ca5fef54a70d45c"
Jul 28 22:49:55.278068 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:49:55.278032    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:49:55.368929 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:49:55.368880    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:50:32.997337 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:50:32.997283    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:51:06.408964 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:51:06.408928    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:51:07.355285 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:51:07.355249    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:51:09.542872 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:51:09.542840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:51:16.845972 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:51:16.845881    2583 scope.go:117] "RemoveContainer" containerID="1e266fde6bcf69ebe5430fd8f764872b22d5a503bdaa3ecb958748c2a1389307"
Jul 28 22:51:16.854759 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:51:16.854740    2583 scope.go:117] "RemoveContainer" containerID="5bd810ffccd5422935270d074971dbcbc054fb65db7e3f209f47620bb7d4577c"
Jul 28 22:51:16.862243 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:51:16.862219    2583 scope.go:117] "RemoveContainer" containerID="55acb90cea2b212ca11217919bd8b2c9ed6c0b539ad89f17aeb7a1d957161abf"
Jul 28 22:51:30.097065 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:51:30.097033    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:51:46.526011 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:51:46.525977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:52:49.226855 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:52:49.226817    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:52:50.139981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:52:50.139942    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:53:00.803979 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:00.803939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:53:23.691570 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.691528    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"]
Jul 28 22:53:23.695053 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.695033    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.697002 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.696977    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1661d9\"/\"kube-root-ca.crt\""
Jul 28 22:53:23.697132 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.696977    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1661d9\"/\"openshift-service-ca.crt\""
Jul 28 22:53:23.697432 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.697415    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1661d9\"/\"default-dockercfg-fhr6m\""
Jul 28 22:53:23.697500 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.697449    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1661d9\"/\"rate-test-model-1661d9-kserve-self-signed-certs\""
Jul 28 22:53:23.705214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.705191    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"]
Jul 28 22:53:23.727373 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.727344    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:53:23.848977 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.848942    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh6wj\" (UniqueName: \"kubernetes.io/projected/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-kube-api-access-lh6wj\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.849157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.849003    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-home\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.849157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.849075    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-tmp-dir\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.849157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.849116    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-model-cache\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.849316 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.849161    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-dshm\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.849316 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.849212    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-tls-certs\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.949905 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.949833    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-home\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.949905 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.949866    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-tmp-dir\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.949905 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.949890    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-model-cache\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.950166 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.949938    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-dshm\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.950166 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.949977    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-tls-certs\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.950166 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.950060    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh6wj\" (UniqueName: \"kubernetes.io/projected/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-kube-api-access-lh6wj\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.950336 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.950251    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-home\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.950336 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.950283    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-tmp-dir\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.950440 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.950347    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-model-cache\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.952230 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.952213    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-dshm\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.952480 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.952463    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-tls-certs\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:23.956994 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:23.956973    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh6wj\" (UniqueName: \"kubernetes.io/projected/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-kube-api-access-lh6wj\") pod \"rate-test-model-1661d9-kserve-84d4485b67-285zq\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") " pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:24.005516 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:24.005493    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:24.141040 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:24.141017    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"]
Jul 28 22:53:24.142831 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:53:24.142806    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod211c7c0b_596a_4d08_a0f7_4d892a9dfd07.slice/crio-f5ad0b3a747cd0a3761423cc4341b9a2ab18232561c05e8b54caa89bfc433c87 WatchSource:0}: Error finding container f5ad0b3a747cd0a3761423cc4341b9a2ab18232561c05e8b54caa89bfc433c87: Status 404 returned error can't find the container with id f5ad0b3a747cd0a3761423cc4341b9a2ab18232561c05e8b54caa89bfc433c87
Jul 28 22:53:24.997517 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:24.997483    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq" event={"ID":"211c7c0b-596a-4d08-a0f7-4d892a9dfd07","Type":"ContainerStarted","Data":"576ed73aafdaeb4647ed33944a1785375d91e01bc391d832976e8a581eb4cc03"}
Jul 28 22:53:24.997954 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:24.997525    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq" event={"ID":"211c7c0b-596a-4d08-a0f7-4d892a9dfd07","Type":"ContainerStarted","Data":"f5ad0b3a747cd0a3761423cc4341b9a2ab18232561c05e8b54caa89bfc433c87"}
Jul 28 22:53:25.015013 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:25.014967    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq" podStartSLOduration=2.014953477 podStartE2EDuration="2.014953477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:53:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:53:25.013296977 +0000 UTC m=+2949.021162544" watchObservedRunningTime="2026-07-28 22:53:25.014953477 +0000 UTC m=+2949.022819035"
Jul 28 22:53:34.005857 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:34.005810    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:34.006340 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:34.005871    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:34.018410 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:34.018388    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:53:34.061106 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:53:34.061078    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:54:16.654947 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:16.654877    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:54:16.679937 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:16.679912    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:54:34.003753 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:34.003712    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:54:34.893581 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:34.893548    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"]
Jul 28 22:54:34.912069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:34.912031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:54:35.976604 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:35.976565    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7594-140116051196080"]
Jul 28 22:54:35.980148 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:35.980115    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051196080"
Jul 28 22:54:35.982006 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:35.981984    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4rblr\""
Jul 28 22:54:35.982988 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:35.982961    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7594-140116051196080"]
Jul 28 22:54:36.078959 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:36.078924    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xln2c\" (UniqueName: \"kubernetes.io/projected/4e2a7572-bbdc-4873-9320-f53ed88473bc-kube-api-access-xln2c\") pod \"test-curl-7594-140116051196080\" (UID: \"4e2a7572-bbdc-4873-9320-f53ed88473bc\") " pod="opendatahub/test-curl-7594-140116051196080"
Jul 28 22:54:36.179387 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:36.179352    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xln2c\" (UniqueName: \"kubernetes.io/projected/4e2a7572-bbdc-4873-9320-f53ed88473bc-kube-api-access-xln2c\") pod \"test-curl-7594-140116051196080\" (UID: \"4e2a7572-bbdc-4873-9320-f53ed88473bc\") " pod="opendatahub/test-curl-7594-140116051196080"
Jul 28 22:54:36.188343 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:36.188316    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xln2c\" (UniqueName: \"kubernetes.io/projected/4e2a7572-bbdc-4873-9320-f53ed88473bc-kube-api-access-xln2c\") pod \"test-curl-7594-140116051196080\" (UID: \"4e2a7572-bbdc-4873-9320-f53ed88473bc\") " pod="opendatahub/test-curl-7594-140116051196080"
Jul 28 22:54:36.289483 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:36.289412    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051196080"
Jul 28 22:54:36.419346 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:36.419322    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7594-140116051196080"]
Jul 28 22:54:36.420886 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:54:36.420852    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e2a7572_bbdc_4873_9320_f53ed88473bc.slice/crio-be863f44a47827416e187bf7c0a406f5cab36464a2df8e4236c00303aba531cc WatchSource:0}: Error finding container be863f44a47827416e187bf7c0a406f5cab36464a2df8e4236c00303aba531cc: Status 404 returned error can't find the container with id be863f44a47827416e187bf7c0a406f5cab36464a2df8e4236c00303aba531cc
Jul 28 22:54:36.422160 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:36.422143    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:54:37.383847 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:37.383739    2583 generic.go:356] "Generic (PLEG): container finished" podID="4e2a7572-bbdc-4873-9320-f53ed88473bc" containerID="a19a6d17b1013252faa9718e27240f4a554eecde964c0644959980208b29e104" exitCode=0
Jul 28 22:54:37.383847 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:37.383787    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7594-140116051196080" event={"ID":"4e2a7572-bbdc-4873-9320-f53ed88473bc","Type":"ContainerDied","Data":"a19a6d17b1013252faa9718e27240f4a554eecde964c0644959980208b29e104"}
Jul 28 22:54:37.383847 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:37.383847    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7594-140116051196080" event={"ID":"4e2a7572-bbdc-4873-9320-f53ed88473bc","Type":"ContainerStarted","Data":"be863f44a47827416e187bf7c0a406f5cab36464a2df8e4236c00303aba531cc"}
Jul 28 22:54:38.530249 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.530222    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051196080"
Jul 28 22:54:38.548310 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.548239    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7594-140116051196080_4e2a7572-bbdc-4873-9320-f53ed88473bc/test-curl-7594-140116051196080/0.log"
Jul 28 22:54:38.571036 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.571002    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7594-140116051196080"]
Jul 28 22:54:38.578036 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.578010    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7594-140116051196080"]
Jul 28 22:54:38.603168 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.603142    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xln2c\" (UniqueName: \"kubernetes.io/projected/4e2a7572-bbdc-4873-9320-f53ed88473bc-kube-api-access-xln2c\") pod \"4e2a7572-bbdc-4873-9320-f53ed88473bc\" (UID: \"4e2a7572-bbdc-4873-9320-f53ed88473bc\") "
Jul 28 22:54:38.605289 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.605269    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e2a7572-bbdc-4873-9320-f53ed88473bc-kube-api-access-xln2c" (OuterVolumeSpecName: "kube-api-access-xln2c") pod "4e2a7572-bbdc-4873-9320-f53ed88473bc" (UID: "4e2a7572-bbdc-4873-9320-f53ed88473bc"). InnerVolumeSpecName "kube-api-access-xln2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:54:38.704328 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.704247    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xln2c\" (UniqueName: \"kubernetes.io/projected/4e2a7572-bbdc-4873-9320-f53ed88473bc-kube-api-access-xln2c\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:54:38.713896 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.713869    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7594-140116051194288"]
Jul 28 22:54:38.714226 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.714214    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e2a7572-bbdc-4873-9320-f53ed88473bc" containerName="test-curl-7594-140116051196080"
Jul 28 22:54:38.714274 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.714228    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e2a7572-bbdc-4873-9320-f53ed88473bc" containerName="test-curl-7594-140116051196080"
Jul 28 22:54:38.714310 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.714304    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e2a7572-bbdc-4873-9320-f53ed88473bc" containerName="test-curl-7594-140116051196080"
Jul 28 22:54:38.717344 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.717328    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051194288"
Jul 28 22:54:38.723399 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.723376    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7594-140116051194288"]
Jul 28 22:54:38.804950 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.804924    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ftp2\" (UniqueName: \"kubernetes.io/projected/4accd9b4-59a1-4959-942f-3928c4f02694-kube-api-access-4ftp2\") pod \"test-curl-7594-140116051194288\" (UID: \"4accd9b4-59a1-4959-942f-3928c4f02694\") " pod="opendatahub/test-curl-7594-140116051194288"
Jul 28 22:54:38.905870 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.905833    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ftp2\" (UniqueName: \"kubernetes.io/projected/4accd9b4-59a1-4959-942f-3928c4f02694-kube-api-access-4ftp2\") pod \"test-curl-7594-140116051194288\" (UID: \"4accd9b4-59a1-4959-942f-3928c4f02694\") " pod="opendatahub/test-curl-7594-140116051194288"
Jul 28 22:54:38.913573 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:38.913551    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ftp2\" (UniqueName: \"kubernetes.io/projected/4accd9b4-59a1-4959-942f-3928c4f02694-kube-api-access-4ftp2\") pod \"test-curl-7594-140116051194288\" (UID: \"4accd9b4-59a1-4959-942f-3928c4f02694\") " pod="opendatahub/test-curl-7594-140116051194288"
Jul 28 22:54:39.026719 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:39.026646    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051194288"
Jul 28 22:54:39.363094 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:39.363068    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7594-140116051194288"]
Jul 28 22:54:39.364650 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:54:39.364622    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4accd9b4_59a1_4959_942f_3928c4f02694.slice/crio-7acea05d60133d665509e9162d605964ce14dfac86f8d7e3b4a542151e86ad04 WatchSource:0}: Error finding container 7acea05d60133d665509e9162d605964ce14dfac86f8d7e3b4a542151e86ad04: Status 404 returned error can't find the container with id 7acea05d60133d665509e9162d605964ce14dfac86f8d7e3b4a542151e86ad04
Jul 28 22:54:39.397258 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:39.397230    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7594-140116051194288" event={"ID":"4accd9b4-59a1-4959-942f-3928c4f02694","Type":"ContainerStarted","Data":"7acea05d60133d665509e9162d605964ce14dfac86f8d7e3b4a542151e86ad04"}
Jul 28 22:54:39.398629 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:39.398612    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051196080"
Jul 28 22:54:39.398735 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:39.398613    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be863f44a47827416e187bf7c0a406f5cab36464a2df8e4236c00303aba531cc"
Jul 28 22:54:39.409286 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:54:39.409260    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7594-140116051196080\" is forbidden: User \"system:node:ip-10-0-128-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-9.ec2.internal' and this object" podUID="4e2a7572-bbdc-4873-9320-f53ed88473bc" pod="opendatahub/test-curl-7594-140116051196080"
Jul 28 22:54:40.406080 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:40.406044    2583 generic.go:356] "Generic (PLEG): container finished" podID="4accd9b4-59a1-4959-942f-3928c4f02694" containerID="50f8240ea5768fb1063caeb21356e015ca55bcb743cfdbb54d8d0aa0df8249e8" exitCode=0
Jul 28 22:54:40.406463 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:40.406093    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7594-140116051194288" event={"ID":"4accd9b4-59a1-4959-942f-3928c4f02694","Type":"ContainerDied","Data":"50f8240ea5768fb1063caeb21356e015ca55bcb743cfdbb54d8d0aa0df8249e8"}
Jul 28 22:54:40.420741 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:54:40.420705    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7594-140116051196080\" is forbidden: User \"system:node:ip-10-0-128-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-9.ec2.internal' and this object" podUID="4e2a7572-bbdc-4873-9320-f53ed88473bc" pod="opendatahub/test-curl-7594-140116051196080"
Jul 28 22:54:40.431737 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:40.431712    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e2a7572-bbdc-4873-9320-f53ed88473bc" path="/var/lib/kubelet/pods/4e2a7572-bbdc-4873-9320-f53ed88473bc/volumes"
Jul 28 22:54:41.549172 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.549149    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051194288"
Jul 28 22:54:41.568045 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.568018    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7594-140116051194288_4accd9b4-59a1-4959-942f-3928c4f02694/test-curl-7594-140116051194288/0.log"
Jul 28 22:54:41.594601 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.594565    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7594-140116051194288"]
Jul 28 22:54:41.596844 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.596814    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7594-140116051194288"]
Jul 28 22:54:41.629221 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.629192    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ftp2\" (UniqueName: \"kubernetes.io/projected/4accd9b4-59a1-4959-942f-3928c4f02694-kube-api-access-4ftp2\") pod \"4accd9b4-59a1-4959-942f-3928c4f02694\" (UID: \"4accd9b4-59a1-4959-942f-3928c4f02694\") "
Jul 28 22:54:41.631194 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.631171    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4accd9b4-59a1-4959-942f-3928c4f02694-kube-api-access-4ftp2" (OuterVolumeSpecName: "kube-api-access-4ftp2") pod "4accd9b4-59a1-4959-942f-3928c4f02694" (UID: "4accd9b4-59a1-4959-942f-3928c4f02694"). InnerVolumeSpecName "kube-api-access-4ftp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:54:41.730412 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.730341    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ftp2\" (UniqueName: \"kubernetes.io/projected/4accd9b4-59a1-4959-942f-3928c4f02694-kube-api-access-4ftp2\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:54:41.760383 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.760350    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7594-140116051417104"]
Jul 28 22:54:41.760717 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.760704    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4accd9b4-59a1-4959-942f-3928c4f02694" containerName="test-curl-7594-140116051194288"
Jul 28 22:54:41.760756 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.760718    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4accd9b4-59a1-4959-942f-3928c4f02694" containerName="test-curl-7594-140116051194288"
Jul 28 22:54:41.760817 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.760790    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4accd9b4-59a1-4959-942f-3928c4f02694" containerName="test-curl-7594-140116051194288"
Jul 28 22:54:41.763898 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.763871    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:41.767162 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.767139    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7594-140116051417104"]
Jul 28 22:54:41.831608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.831580    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzcgl\" (UniqueName: \"kubernetes.io/projected/51f2b226-a5da-4008-9d19-b768fe864218-kube-api-access-fzcgl\") pod \"test-curl-7594-140116051417104\" (UID: \"51f2b226-a5da-4008-9d19-b768fe864218\") " pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:41.932375 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.932348    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzcgl\" (UniqueName: \"kubernetes.io/projected/51f2b226-a5da-4008-9d19-b768fe864218-kube-api-access-fzcgl\") pod \"test-curl-7594-140116051417104\" (UID: \"51f2b226-a5da-4008-9d19-b768fe864218\") " pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:41.940128 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:41.940099    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzcgl\" (UniqueName: \"kubernetes.io/projected/51f2b226-a5da-4008-9d19-b768fe864218-kube-api-access-fzcgl\") pod \"test-curl-7594-140116051417104\" (UID: \"51f2b226-a5da-4008-9d19-b768fe864218\") " pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:42.074092 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:42.074004    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:42.203735 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:42.203710    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7594-140116051417104"]
Jul 28 22:54:42.204937 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:54:42.204911    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51f2b226_a5da_4008_9d19_b768fe864218.slice/crio-3025ac7c812acf7dc467fde58720942e7beea56b594ea5de3ba926552c4b71f9 WatchSource:0}: Error finding container 3025ac7c812acf7dc467fde58720942e7beea56b594ea5de3ba926552c4b71f9: Status 404 returned error can't find the container with id 3025ac7c812acf7dc467fde58720942e7beea56b594ea5de3ba926552c4b71f9
Jul 28 22:54:42.417292 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:42.417255    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7594-140116051417104" event={"ID":"51f2b226-a5da-4008-9d19-b768fe864218","Type":"ContainerStarted","Data":"3025ac7c812acf7dc467fde58720942e7beea56b594ea5de3ba926552c4b71f9"}
Jul 28 22:54:42.418479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:42.418457    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7acea05d60133d665509e9162d605964ce14dfac86f8d7e3b4a542151e86ad04"
Jul 28 22:54:42.418479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:42.418474    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051194288"
Jul 28 22:54:42.428660 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:54:42.428637    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7594-140116051194288\" is forbidden: User \"system:node:ip-10-0-128-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-9.ec2.internal' and this object" podUID="4accd9b4-59a1-4959-942f-3928c4f02694" pod="opendatahub/test-curl-7594-140116051194288"
Jul 28 22:54:42.431354 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:42.431336    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4accd9b4-59a1-4959-942f-3928c4f02694" path="/var/lib/kubelet/pods/4accd9b4-59a1-4959-942f-3928c4f02694/volumes"
Jul 28 22:54:43.426518 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:43.426482    2583 generic.go:356] "Generic (PLEG): container finished" podID="51f2b226-a5da-4008-9d19-b768fe864218" containerID="da1d770b33a27b282aa55c89caccb556cb650edf6eb63af8c0121d3936a9a2d5" exitCode=0
Jul 28 22:54:43.426916 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:43.426550    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7594-140116051417104" event={"ID":"51f2b226-a5da-4008-9d19-b768fe864218","Type":"ContainerDied","Data":"da1d770b33a27b282aa55c89caccb556cb650edf6eb63af8c0121d3936a9a2d5"}
Jul 28 22:54:44.565886 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.565864    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:44.605505 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.605480    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7594-140116051417104_51f2b226-a5da-4008-9d19-b768fe864218/test-curl-7594-140116051417104/0.log"
Jul 28 22:54:44.655050 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.655018    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7594-140116051417104"]
Jul 28 22:54:44.655304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.655286    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzcgl\" (UniqueName: \"kubernetes.io/projected/51f2b226-a5da-4008-9d19-b768fe864218-kube-api-access-fzcgl\") pod \"51f2b226-a5da-4008-9d19-b768fe864218\" (UID: \"51f2b226-a5da-4008-9d19-b768fe864218\") "
Jul 28 22:54:44.657306 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.657282    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51f2b226-a5da-4008-9d19-b768fe864218-kube-api-access-fzcgl" (OuterVolumeSpecName: "kube-api-access-fzcgl") pod "51f2b226-a5da-4008-9d19-b768fe864218" (UID: "51f2b226-a5da-4008-9d19-b768fe864218"). InnerVolumeSpecName "kube-api-access-fzcgl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:54:44.667906 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.667882    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7594-140116051417104"]
Jul 28 22:54:44.756343 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.756286    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzcgl\" (UniqueName: \"kubernetes.io/projected/51f2b226-a5da-4008-9d19-b768fe864218-kube-api-access-fzcgl\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:54:44.806340 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.806314    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7594-140116051417104"]
Jul 28 22:54:44.806737 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.806717    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51f2b226-a5da-4008-9d19-b768fe864218" containerName="test-curl-7594-140116051417104"
Jul 28 22:54:44.806737 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.806734    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="51f2b226-a5da-4008-9d19-b768fe864218" containerName="test-curl-7594-140116051417104"
Jul 28 22:54:44.806893 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.806820    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="51f2b226-a5da-4008-9d19-b768fe864218" containerName="test-curl-7594-140116051417104"
Jul 28 22:54:44.809835 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.809819    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:44.816281 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.816260    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7594-140116051417104"]
Jul 28 22:54:44.857458 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.857433    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7c82\" (UniqueName: \"kubernetes.io/projected/86c7e83b-db8e-4b2e-85d1-751758835128-kube-api-access-m7c82\") pod \"test-curl-7594-140116051417104\" (UID: \"86c7e83b-db8e-4b2e-85d1-751758835128\") " pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:44.958627 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.958603    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7c82\" (UniqueName: \"kubernetes.io/projected/86c7e83b-db8e-4b2e-85d1-751758835128-kube-api-access-m7c82\") pod \"test-curl-7594-140116051417104\" (UID: \"86c7e83b-db8e-4b2e-85d1-751758835128\") " pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:44.965513 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:44.965494    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7c82\" (UniqueName: \"kubernetes.io/projected/86c7e83b-db8e-4b2e-85d1-751758835128-kube-api-access-m7c82\") pod \"test-curl-7594-140116051417104\" (UID: \"86c7e83b-db8e-4b2e-85d1-751758835128\") " pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:45.119698 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:45.119673    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:45.443837 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:45.441836    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3025ac7c812acf7dc467fde58720942e7beea56b594ea5de3ba926552c4b71f9"
Jul 28 22:54:45.443837 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:45.441894    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:45.454695 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:45.454662    2583 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7594-140116051417104" oldPodUID="51f2b226-a5da-4008-9d19-b768fe864218" podUID="86c7e83b-db8e-4b2e-85d1-751758835128"
Jul 28 22:54:45.457736 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:45.457717    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7594-140116051417104"]
Jul 28 22:54:45.459306 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:54:45.459287    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86c7e83b_db8e_4b2e_85d1_751758835128.slice/crio-2c2551a926418cfa9d0ab4c84db31b7deb3b8a15f21f246efb612d0376c16b8e WatchSource:0}: Error finding container 2c2551a926418cfa9d0ab4c84db31b7deb3b8a15f21f246efb612d0376c16b8e: Status 404 returned error can't find the container with id 2c2551a926418cfa9d0ab4c84db31b7deb3b8a15f21f246efb612d0376c16b8e
Jul 28 22:54:46.436719 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:46.436684    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51f2b226-a5da-4008-9d19-b768fe864218" path="/var/lib/kubelet/pods/51f2b226-a5da-4008-9d19-b768fe864218/volumes"
Jul 28 22:54:46.448635 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:46.448611    2583 generic.go:356] "Generic (PLEG): container finished" podID="86c7e83b-db8e-4b2e-85d1-751758835128" containerID="c03d99b669f111c9cd436b863384cce0b6f8f14d620cb7f6e2b91c7f119a8c03" exitCode=0
Jul 28 22:54:46.448755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:46.448692    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7594-140116051417104" event={"ID":"86c7e83b-db8e-4b2e-85d1-751758835128","Type":"ContainerDied","Data":"c03d99b669f111c9cd436b863384cce0b6f8f14d620cb7f6e2b91c7f119a8c03"}
Jul 28 22:54:46.448755 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:46.448725    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7594-140116051417104" event={"ID":"86c7e83b-db8e-4b2e-85d1-751758835128","Type":"ContainerStarted","Data":"2c2551a926418cfa9d0ab4c84db31b7deb3b8a15f21f246efb612d0376c16b8e"}
Jul 28 22:54:47.590014 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.589989    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:47.608946 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.608924    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7594-140116051417104_86c7e83b-db8e-4b2e-85d1-751758835128/test-curl-7594-140116051417104/0.log"
Jul 28 22:54:47.634499 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.634470    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7594-140116051417104"]
Jul 28 22:54:47.638239 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.638220    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7594-140116051417104"]
Jul 28 22:54:47.681491 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.681470    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7c82\" (UniqueName: \"kubernetes.io/projected/86c7e83b-db8e-4b2e-85d1-751758835128-kube-api-access-m7c82\") pod \"86c7e83b-db8e-4b2e-85d1-751758835128\" (UID: \"86c7e83b-db8e-4b2e-85d1-751758835128\") "
Jul 28 22:54:47.683431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.683409    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86c7e83b-db8e-4b2e-85d1-751758835128-kube-api-access-m7c82" (OuterVolumeSpecName: "kube-api-access-m7c82") pod "86c7e83b-db8e-4b2e-85d1-751758835128" (UID: "86c7e83b-db8e-4b2e-85d1-751758835128"). InnerVolumeSpecName "kube-api-access-m7c82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:54:47.754280 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.754210    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7594-140116051418112"]
Jul 28 22:54:47.754633 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.754615    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86c7e83b-db8e-4b2e-85d1-751758835128" containerName="test-curl-7594-140116051417104"
Jul 28 22:54:47.754713 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.754636    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="86c7e83b-db8e-4b2e-85d1-751758835128" containerName="test-curl-7594-140116051417104"
Jul 28 22:54:47.754774 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.754755    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="86c7e83b-db8e-4b2e-85d1-751758835128" containerName="test-curl-7594-140116051417104"
Jul 28 22:54:47.757715 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.757698    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:47.764098 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.764078    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7594-140116051418112"]
Jul 28 22:54:47.782330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.782306    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7c82\" (UniqueName: \"kubernetes.io/projected/86c7e83b-db8e-4b2e-85d1-751758835128-kube-api-access-m7c82\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:54:47.883402 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.883373    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4mlg\" (UniqueName: \"kubernetes.io/projected/05c09702-7544-46ee-b4d6-8da70a399b6a-kube-api-access-s4mlg\") pod \"test-curl-7594-140116051418112\" (UID: \"05c09702-7544-46ee-b4d6-8da70a399b6a\") " pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:47.984052 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.984021    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4mlg\" (UniqueName: \"kubernetes.io/projected/05c09702-7544-46ee-b4d6-8da70a399b6a-kube-api-access-s4mlg\") pod \"test-curl-7594-140116051418112\" (UID: \"05c09702-7544-46ee-b4d6-8da70a399b6a\") " pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:47.991776 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:47.991752    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4mlg\" (UniqueName: \"kubernetes.io/projected/05c09702-7544-46ee-b4d6-8da70a399b6a-kube-api-access-s4mlg\") pod \"test-curl-7594-140116051418112\" (UID: \"05c09702-7544-46ee-b4d6-8da70a399b6a\") " pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:48.067891 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:48.067779    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:48.195409 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:48.195380    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7594-140116051418112"]
Jul 28 22:54:48.196775 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:54:48.196734    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05c09702_7544_46ee_b4d6_8da70a399b6a.slice/crio-e6bd29e6b192d45e54c7a640fa394f61308896c8328c2108eb4bdfecb99f60c3 WatchSource:0}: Error finding container e6bd29e6b192d45e54c7a640fa394f61308896c8328c2108eb4bdfecb99f60c3: Status 404 returned error can't find the container with id e6bd29e6b192d45e54c7a640fa394f61308896c8328c2108eb4bdfecb99f60c3
Jul 28 22:54:48.431543 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:48.431512    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86c7e83b-db8e-4b2e-85d1-751758835128" path="/var/lib/kubelet/pods/86c7e83b-db8e-4b2e-85d1-751758835128/volumes"
Jul 28 22:54:48.461214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:48.461179    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7594-140116051418112" event={"ID":"05c09702-7544-46ee-b4d6-8da70a399b6a","Type":"ContainerStarted","Data":"e6bd29e6b192d45e54c7a640fa394f61308896c8328c2108eb4bdfecb99f60c3"}
Jul 28 22:54:48.462373 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:48.462354    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051417104"
Jul 28 22:54:48.462502 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:48.462412    2583 scope.go:117] "RemoveContainer" containerID="c03d99b669f111c9cd436b863384cce0b6f8f14d620cb7f6e2b91c7f119a8c03"
Jul 28 22:54:49.469751 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:49.469714    2583 generic.go:356] "Generic (PLEG): container finished" podID="05c09702-7544-46ee-b4d6-8da70a399b6a" containerID="2cb7caea24c4f4413195a1e4f3252f9c553a33af2df1daba02a6c2f262f32e64" exitCode=0
Jul 28 22:54:49.470195 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:49.469783    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7594-140116051418112" event={"ID":"05c09702-7544-46ee-b4d6-8da70a399b6a","Type":"ContainerDied","Data":"2cb7caea24c4f4413195a1e4f3252f9c553a33af2df1daba02a6c2f262f32e64"}
Jul 28 22:54:49.912630 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:49.912593    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq" podUID="211c7c0b-596a-4d08-a0f7-4d892a9dfd07" containerName="main" containerID="cri-o://576ed73aafdaeb4647ed33944a1785375d91e01bc391d832976e8a581eb4cc03" gracePeriod=45
Jul 28 22:54:50.173157 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.173102    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:54:50.305017 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.304984    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-home\") pod \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") "
Jul 28 22:54:50.305184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.305077    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh6wj\" (UniqueName: \"kubernetes.io/projected/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-kube-api-access-lh6wj\") pod \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") "
Jul 28 22:54:50.305184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.305102    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-tmp-dir\") pod \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") "
Jul 28 22:54:50.305184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.305118    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-model-cache\") pod \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") "
Jul 28 22:54:50.305184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.305139    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-dshm\") pod \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") "
Jul 28 22:54:50.305184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.305157    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-tls-certs\") pod \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\" (UID: \"211c7c0b-596a-4d08-a0f7-4d892a9dfd07\") "
Jul 28 22:54:50.305450 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.305252    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-home" (OuterVolumeSpecName: "home") pod "211c7c0b-596a-4d08-a0f7-4d892a9dfd07" (UID: "211c7c0b-596a-4d08-a0f7-4d892a9dfd07"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:54:50.305450 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.305350    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-model-cache" (OuterVolumeSpecName: "model-cache") pod "211c7c0b-596a-4d08-a0f7-4d892a9dfd07" (UID: "211c7c0b-596a-4d08-a0f7-4d892a9dfd07"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:54:50.305450 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.305383    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "211c7c0b-596a-4d08-a0f7-4d892a9dfd07" (UID: "211c7c0b-596a-4d08-a0f7-4d892a9dfd07"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:54:50.305450 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.305393    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-home\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:54:50.307481 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.307448    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-kube-api-access-lh6wj" (OuterVolumeSpecName: "kube-api-access-lh6wj") pod "211c7c0b-596a-4d08-a0f7-4d892a9dfd07" (UID: "211c7c0b-596a-4d08-a0f7-4d892a9dfd07"). InnerVolumeSpecName "kube-api-access-lh6wj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:54:50.307589 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.307494    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "211c7c0b-596a-4d08-a0f7-4d892a9dfd07" (UID: "211c7c0b-596a-4d08-a0f7-4d892a9dfd07"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:54:50.307589 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.307494    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-dshm" (OuterVolumeSpecName: "dshm") pod "211c7c0b-596a-4d08-a0f7-4d892a9dfd07" (UID: "211c7c0b-596a-4d08-a0f7-4d892a9dfd07"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:54:50.406072 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.406045    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh6wj\" (UniqueName: \"kubernetes.io/projected/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-kube-api-access-lh6wj\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:54:50.406072 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.406069    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-tmp-dir\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:54:50.406239 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.406079    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-model-cache\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:54:50.406239 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.406087    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-dshm\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:54:50.406239 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.406097    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/211c7c0b-596a-4d08-a0f7-4d892a9dfd07-tls-certs\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:54:50.476560 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.476530    2583 generic.go:356] "Generic (PLEG): container finished" podID="211c7c0b-596a-4d08-a0f7-4d892a9dfd07" containerID="576ed73aafdaeb4647ed33944a1785375d91e01bc391d832976e8a581eb4cc03" exitCode=0
Jul 28 22:54:50.477042 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.476581    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"
Jul 28 22:54:50.477042 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.476623    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq" event={"ID":"211c7c0b-596a-4d08-a0f7-4d892a9dfd07","Type":"ContainerDied","Data":"576ed73aafdaeb4647ed33944a1785375d91e01bc391d832976e8a581eb4cc03"}
Jul 28 22:54:50.477042 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.476664    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq" event={"ID":"211c7c0b-596a-4d08-a0f7-4d892a9dfd07","Type":"ContainerDied","Data":"f5ad0b3a747cd0a3761423cc4341b9a2ab18232561c05e8b54caa89bfc433c87"}
Jul 28 22:54:50.477042 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.476682    2583 scope.go:117] "RemoveContainer" containerID="576ed73aafdaeb4647ed33944a1785375d91e01bc391d832976e8a581eb4cc03"
Jul 28 22:54:50.485639 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.485613    2583 scope.go:117] "RemoveContainer" containerID="576ed73aafdaeb4647ed33944a1785375d91e01bc391d832976e8a581eb4cc03"
Jul 28 22:54:50.485875 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:54:50.485856    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"576ed73aafdaeb4647ed33944a1785375d91e01bc391d832976e8a581eb4cc03\": container with ID starting with 576ed73aafdaeb4647ed33944a1785375d91e01bc391d832976e8a581eb4cc03 not found: ID does not exist" containerID="576ed73aafdaeb4647ed33944a1785375d91e01bc391d832976e8a581eb4cc03"
Jul 28 22:54:50.485956 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.485887    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576ed73aafdaeb4647ed33944a1785375d91e01bc391d832976e8a581eb4cc03"} err="failed to get container status \"576ed73aafdaeb4647ed33944a1785375d91e01bc391d832976e8a581eb4cc03\": rpc error: code = NotFound desc = could not find container \"576ed73aafdaeb4647ed33944a1785375d91e01bc391d832976e8a581eb4cc03\": container with ID starting with 576ed73aafdaeb4647ed33944a1785375d91e01bc391d832976e8a581eb4cc03 not found: ID does not exist"
Jul 28 22:54:50.492720 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.492698    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"]
Jul 28 22:54:50.496100 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.496077    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1661d9/rate-test-model-1661d9-kserve-84d4485b67-285zq"]
Jul 28 22:54:50.596970 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.596948    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:50.615059 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.615003    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7594-140116051418112_05c09702-7544-46ee-b4d6-8da70a399b6a/test-curl-7594-140116051418112/0.log"
Jul 28 22:54:50.638152 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.638121    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7594-140116051418112"]
Jul 28 22:54:50.645542 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.645503    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7594-140116051418112"]
Jul 28 22:54:50.708742 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.708654    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4mlg\" (UniqueName: \"kubernetes.io/projected/05c09702-7544-46ee-b4d6-8da70a399b6a-kube-api-access-s4mlg\") pod \"05c09702-7544-46ee-b4d6-8da70a399b6a\" (UID: \"05c09702-7544-46ee-b4d6-8da70a399b6a\") "
Jul 28 22:54:50.710691 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.710668    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05c09702-7544-46ee-b4d6-8da70a399b6a-kube-api-access-s4mlg" (OuterVolumeSpecName: "kube-api-access-s4mlg") pod "05c09702-7544-46ee-b4d6-8da70a399b6a" (UID: "05c09702-7544-46ee-b4d6-8da70a399b6a"). InnerVolumeSpecName "kube-api-access-s4mlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:54:50.760310 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.760284    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7594-140116051418112"]
Jul 28 22:54:50.760669 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.760657    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="211c7c0b-596a-4d08-a0f7-4d892a9dfd07" containerName="main"
Jul 28 22:54:50.760709 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.760671    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="211c7c0b-596a-4d08-a0f7-4d892a9dfd07" containerName="main"
Jul 28 22:54:50.760709 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.760681    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05c09702-7544-46ee-b4d6-8da70a399b6a" containerName="test-curl-7594-140116051418112"
Jul 28 22:54:50.760709 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.760687    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="05c09702-7544-46ee-b4d6-8da70a399b6a" containerName="test-curl-7594-140116051418112"
Jul 28 22:54:50.760836 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.760764    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="05c09702-7544-46ee-b4d6-8da70a399b6a" containerName="test-curl-7594-140116051418112"
Jul 28 22:54:50.760836 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.760773    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="211c7c0b-596a-4d08-a0f7-4d892a9dfd07" containerName="main"
Jul 28 22:54:50.763768 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.763753    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:50.769711 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.769685    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7594-140116051418112"]
Jul 28 22:54:50.809630 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.809603    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrgtq\" (UniqueName: \"kubernetes.io/projected/9d58f553-551b-4a6e-bb1b-972d49eb6110-kube-api-access-vrgtq\") pod \"test-curl-7594-140116051418112\" (UID: \"9d58f553-551b-4a6e-bb1b-972d49eb6110\") " pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:50.809745 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.809649    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4mlg\" (UniqueName: \"kubernetes.io/projected/05c09702-7544-46ee-b4d6-8da70a399b6a-kube-api-access-s4mlg\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:54:50.910360 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.910332    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrgtq\" (UniqueName: \"kubernetes.io/projected/9d58f553-551b-4a6e-bb1b-972d49eb6110-kube-api-access-vrgtq\") pod \"test-curl-7594-140116051418112\" (UID: \"9d58f553-551b-4a6e-bb1b-972d49eb6110\") " pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:50.917815 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:50.917777    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrgtq\" (UniqueName: \"kubernetes.io/projected/9d58f553-551b-4a6e-bb1b-972d49eb6110-kube-api-access-vrgtq\") pod \"test-curl-7594-140116051418112\" (UID: \"9d58f553-551b-4a6e-bb1b-972d49eb6110\") " pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:51.073467 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:51.073385    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:51.203731 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:51.203707    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7594-140116051418112"]
Jul 28 22:54:51.205619 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:54:51.205591    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d58f553_551b_4a6e_bb1b_972d49eb6110.slice/crio-695f8908256ae23f78389b99bad1ddf9a779e8bb186e2822395b7c5633cc17ec WatchSource:0}: Error finding container 695f8908256ae23f78389b99bad1ddf9a779e8bb186e2822395b7c5633cc17ec: Status 404 returned error can't find the container with id 695f8908256ae23f78389b99bad1ddf9a779e8bb186e2822395b7c5633cc17ec
Jul 28 22:54:51.482876 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:51.482843    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7594-140116051418112" event={"ID":"9d58f553-551b-4a6e-bb1b-972d49eb6110","Type":"ContainerStarted","Data":"695f8908256ae23f78389b99bad1ddf9a779e8bb186e2822395b7c5633cc17ec"}
Jul 28 22:54:51.485333 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:51.485312    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6bd29e6b192d45e54c7a640fa394f61308896c8328c2108eb4bdfecb99f60c3"
Jul 28 22:54:51.485411 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:51.485352    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:51.513377 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:51.513347    2583 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7594-140116051418112" oldPodUID="05c09702-7544-46ee-b4d6-8da70a399b6a" podUID="9d58f553-551b-4a6e-bb1b-972d49eb6110"
Jul 28 22:54:52.431830 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:52.431766    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05c09702-7544-46ee-b4d6-8da70a399b6a" path="/var/lib/kubelet/pods/05c09702-7544-46ee-b4d6-8da70a399b6a/volumes"
Jul 28 22:54:52.432258 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:52.432237    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="211c7c0b-596a-4d08-a0f7-4d892a9dfd07" path="/var/lib/kubelet/pods/211c7c0b-596a-4d08-a0f7-4d892a9dfd07/volumes"
Jul 28 22:54:52.492204 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:52.492168    2583 generic.go:356] "Generic (PLEG): container finished" podID="9d58f553-551b-4a6e-bb1b-972d49eb6110" containerID="5bede72acf3ddde9b5af26e06e636ac5d6157e006647d8b263f927e27684156a" exitCode=0
Jul 28 22:54:52.492544 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:52.492213    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7594-140116051418112" event={"ID":"9d58f553-551b-4a6e-bb1b-972d49eb6110","Type":"ContainerDied","Data":"5bede72acf3ddde9b5af26e06e636ac5d6157e006647d8b263f927e27684156a"}
Jul 28 22:54:53.626241 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:53.626215    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:53.647097 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:53.647070    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7594-140116051418112_9d58f553-551b-4a6e-bb1b-972d49eb6110/test-curl-7594-140116051418112/0.log"
Jul 28 22:54:53.669277 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:53.669247    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7594-140116051418112"]
Jul 28 22:54:53.672339 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:53.672314    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7594-140116051418112"]
Jul 28 22:54:53.733609 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:53.733586    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrgtq\" (UniqueName: \"kubernetes.io/projected/9d58f553-551b-4a6e-bb1b-972d49eb6110-kube-api-access-vrgtq\") pod \"9d58f553-551b-4a6e-bb1b-972d49eb6110\" (UID: \"9d58f553-551b-4a6e-bb1b-972d49eb6110\") "
Jul 28 22:54:53.735580 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:53.735556    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d58f553-551b-4a6e-bb1b-972d49eb6110-kube-api-access-vrgtq" (OuterVolumeSpecName: "kube-api-access-vrgtq") pod "9d58f553-551b-4a6e-bb1b-972d49eb6110" (UID: "9d58f553-551b-4a6e-bb1b-972d49eb6110"). InnerVolumeSpecName "kube-api-access-vrgtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:54:53.835225 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:53.835163    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrgtq\" (UniqueName: \"kubernetes.io/projected/9d58f553-551b-4a6e-bb1b-972d49eb6110-kube-api-access-vrgtq\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:54:54.185161 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.185129    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:54:54.432172 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.432144    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d58f553-551b-4a6e-bb1b-972d49eb6110" path="/var/lib/kubelet/pods/9d58f553-551b-4a6e-bb1b-972d49eb6110/volumes"
Jul 28 22:54:54.476471 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.476443    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"]
Jul 28 22:54:54.476895 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.476877    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d58f553-551b-4a6e-bb1b-972d49eb6110" containerName="test-curl-7594-140116051418112"
Jul 28 22:54:54.477058 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.476896    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d58f553-551b-4a6e-bb1b-972d49eb6110" containerName="test-curl-7594-140116051418112"
Jul 28 22:54:54.477058 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.476980    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d58f553-551b-4a6e-bb1b-972d49eb6110" containerName="test-curl-7594-140116051418112"
Jul 28 22:54:54.480196 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.480177    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.482204 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.482185    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-ff4fa1-openshift-default-dockercfg-64ktq\""
Jul 28 22:54:54.491187 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.491165    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"]
Jul 28 22:54:54.501586 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.501559    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:54:54.507046 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.507029    2583 scope.go:117] "RemoveContainer" containerID="5bede72acf3ddde9b5af26e06e636ac5d6157e006647d8b263f927e27684156a"
Jul 28 22:54:54.507046 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.507040    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7594-140116051418112"
Jul 28 22:54:54.540440 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.540418    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc664d93-a103-4deb-b939-d312a1f435a1-istiod-ca-cert\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.540548 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.540452    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-workload-socket\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.540548 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.540471    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76kq9\" (UniqueName: \"kubernetes.io/projected/cc664d93-a103-4deb-b939-d312a1f435a1-kube-api-access-76kq9\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.540548 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.540504    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-credential-socket\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.540548 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.540523    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc664d93-a103-4deb-b939-d312a1f435a1-istio-token\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.540695 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.540561    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-workload-certs\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.540695 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.540628    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-istio-envoy\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.540695 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.540653    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc664d93-a103-4deb-b939-d312a1f435a1-istio-podinfo\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.540695 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.540677    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-istio-data\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.641829 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.641788    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-istio-envoy\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.641829 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.641838    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc664d93-a103-4deb-b939-d312a1f435a1-istio-podinfo\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.642330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.641951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-istio-data\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.642330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.641989    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc664d93-a103-4deb-b939-d312a1f435a1-istiod-ca-cert\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.642330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.642027    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-workload-socket\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.642330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.642050    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76kq9\" (UniqueName: \"kubernetes.io/projected/cc664d93-a103-4deb-b939-d312a1f435a1-kube-api-access-76kq9\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.642330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.642095    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-credential-socket\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.642330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.642119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc664d93-a103-4deb-b939-d312a1f435a1-istio-token\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.642330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.642148    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-workload-certs\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.642330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.642309    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-istio-data\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.642765 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.642372    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-workload-socket\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.642765 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.642542    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-credential-socket\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.643123 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.642891    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc664d93-a103-4deb-b939-d312a1f435a1-istiod-ca-cert\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.643316 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.643095    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-workload-certs\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.651366 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.651341    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-istio-envoy\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.652224 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.652193    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc664d93-a103-4deb-b939-d312a1f435a1-istio-podinfo\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.653931 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.653908    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc664d93-a103-4deb-b939-d312a1f435a1-istio-token\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.654245 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.654225    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76kq9\" (UniqueName: \"kubernetes.io/projected/cc664d93-a103-4deb-b939-d312a1f435a1-kube-api-access-76kq9\") pod \"e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") " pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.791990 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.791911    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:54.927270 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.927245    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"]
Jul 28 22:54:54.928936 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:54:54.928911    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc664d93_a103_4deb_b939_d312a1f435a1.slice/crio-d74584b48a5e54d2e9443d139ba8933b45be9f044e24abb73022e58cdda5568b WatchSource:0}: Error finding container d74584b48a5e54d2e9443d139ba8933b45be9f044e24abb73022e58cdda5568b: Status 404 returned error can't find the container with id d74584b48a5e54d2e9443d139ba8933b45be9f044e24abb73022e58cdda5568b
Jul 28 22:54:54.931611 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.931580    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:54:54.931675 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.931660    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:54:54.931720 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:54.931689    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:54:55.112064 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:55.111965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:54:55.515199 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:55.515165    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" event={"ID":"cc664d93-a103-4deb-b939-d312a1f435a1","Type":"ContainerStarted","Data":"4c3ad29bb0b9b23233b2c09cd2dcef9bbb1a5232aa7d3d2fb19a1a288ef076d1"}
Jul 28 22:54:55.515360 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:55.515228    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" event={"ID":"cc664d93-a103-4deb-b939-d312a1f435a1","Type":"ContainerStarted","Data":"d74584b48a5e54d2e9443d139ba8933b45be9f044e24abb73022e58cdda5568b"}
Jul 28 22:54:55.535022 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:55.534972    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" podStartSLOduration=1.5349568740000001 podStartE2EDuration="1.534956874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:54:55.532498206 +0000 UTC m=+3039.540363763" watchObservedRunningTime="2026-07-28 22:54:55.534956874 +0000 UTC m=+3039.542822430"
Jul 28 22:54:55.792989 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:55.792914    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:55.798034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:55.798013    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:56.520685 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:56.520653    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:56.521538 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:56.521521    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:54:57.461742 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:57.461702    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:54:57.562670 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:57.562636    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:54:57.729446 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:57.729365    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:54:58.116775 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:58.116748    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:54:58.355065 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:58.355026    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:54:58.453876 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:54:58.453766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:12.563012 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:12.562964    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 28 22:55:12.563550 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:12.563048    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 28 22:55:14.622084 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:14.622050    2583 generic.go:356] "Generic (PLEG): container finished" podID="cc664d93-a103-4deb-b939-d312a1f435a1" containerID="4c3ad29bb0b9b23233b2c09cd2dcef9bbb1a5232aa7d3d2fb19a1a288ef076d1" exitCode=0
Jul 28 22:55:14.622488 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:14.622118    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" event={"ID":"cc664d93-a103-4deb-b939-d312a1f435a1","Type":"ContainerDied","Data":"4c3ad29bb0b9b23233b2c09cd2dcef9bbb1a5232aa7d3d2fb19a1a288ef076d1"}
Jul 28 22:55:14.622543 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:14.622494    2583 scope.go:117] "RemoveContainer" containerID="4c3ad29bb0b9b23233b2c09cd2dcef9bbb1a5232aa7d3d2fb19a1a288ef076d1"
Jul 28 22:55:14.623501 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:14.623476    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:55:14.623611 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:14.623538    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:55:14.623611 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:14.623570    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:55:15.629861 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:15.629828    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" event={"ID":"cc664d93-a103-4deb-b939-d312a1f435a1","Type":"ContainerStarted","Data":"b80ff847bbd90bd96ad4f92c2ba1ecf038b8dad929eafc8a95e44f8026d42eaa"}
Jul 28 22:55:15.792898 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:15.792858    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:55:15.793118 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:15.793091    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Jul 28 22:55:15.793197 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:15.793163    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Jul 28 22:55:16.792698 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:16.792611    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Jul 28 22:55:16.792698 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:16.792661    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Jul 28 22:55:17.792434 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:17.792398    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Jul 28 22:55:17.792617 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:17.792452    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Jul 28 22:55:18.792501 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:18.792461    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Jul 28 22:55:18.792915 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:18.792521    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Jul 28 22:55:19.792984 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:19.792949    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Jul 28 22:55:19.793375 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:19.793002    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Jul 28 22:55:20.795677 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:20.795649    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:55:20.796060 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:20.795915    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:55:20.796519 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:20.796505    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:55:21.172515 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:21.172483    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:21.252409 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:21.252384    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:22.448838 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:22.448787    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:23.139066 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.139032    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:23.436471 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.436397    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"]
Jul 28 22:55:23.441724 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.441704    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.444151 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.444130    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-056c8a-openshift-default-dockercfg-rzw5q\""
Jul 28 22:55:23.455049 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.455022    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"]
Jul 28 22:55:23.471866 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.471838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:23.512920 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.512887    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv7nr\" (UniqueName: \"kubernetes.io/projected/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-kube-api-access-fv7nr\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.513069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.512937    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-workload-socket\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.513069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.512958    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-envoy\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.513069 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.513016    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-data\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.513210 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.513069    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-credential-socket\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.513210 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.513093    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-token\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.513210 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.513123    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istiod-ca-cert\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.513347 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.513214    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-workload-certs\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.513402 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.513373    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-podinfo\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.614531 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.614490    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-workload-certs\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.614718 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.614541    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-podinfo\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.614718 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.614574    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv7nr\" (UniqueName: \"kubernetes.io/projected/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-kube-api-access-fv7nr\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.614718 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.614598    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-workload-socket\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.614718 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.614613    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-envoy\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.614718 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.614656    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-data\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.614718 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.614699    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-credential-socket\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.615155 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.614722    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-token\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.615155 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.614750    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istiod-ca-cert\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.615155 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.614974    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-workload-certs\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.615155 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.615042    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-workload-socket\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.615371 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.615212    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-credential-socket\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.615371 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.615359    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-data\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.615482 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.615460    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istiod-ca-cert\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.616952 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.616931    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-envoy\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.617091 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.617074    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-podinfo\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.622528 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.622503    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv7nr\" (UniqueName: \"kubernetes.io/projected/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-kube-api-access-fv7nr\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.622653 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.622636    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-token\") pod \"e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") " pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.752497 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.752406    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:23.805852 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.805777    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:23.897707 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.897671    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"]
Jul 28 22:55:23.900991 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:55:23.900963    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaea64ae1_5d5a_44e7_8f8f_4c397c7533dc.slice/crio-0d9fb46baf9e017c0ea587bc9821201c0fe3a1206c5a9c0d4ab83cf88da6d731 WatchSource:0}: Error finding container 0d9fb46baf9e017c0ea587bc9821201c0fe3a1206c5a9c0d4ab83cf88da6d731: Status 404 returned error can't find the container with id 0d9fb46baf9e017c0ea587bc9821201c0fe3a1206c5a9c0d4ab83cf88da6d731
Jul 28 22:55:23.903025 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.902994    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:55:23.903129 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.903070    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:55:23.903129 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:23.903113    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:55:24.690388 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:24.690357    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29" event={"ID":"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc","Type":"ContainerStarted","Data":"1236a90a202e75d47f1eb7d290aa166f67b0e4d680d7646ef49e97a2955945d6"}
Jul 28 22:55:24.690834 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:24.690407    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29" event={"ID":"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc","Type":"ContainerStarted","Data":"0d9fb46baf9e017c0ea587bc9821201c0fe3a1206c5a9c0d4ab83cf88da6d731"}
Jul 28 22:55:24.714488 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:24.714440    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29" podStartSLOduration=1.7144276710000002 podStartE2EDuration="1.714427671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:55:24.711593682 +0000 UTC m=+3068.719459241" watchObservedRunningTime="2026-07-28 22:55:24.714427671 +0000 UTC m=+3068.722293229"
Jul 28 22:55:24.753055 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:24.753032    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:24.757850 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:24.757831    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:25.695538 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:25.695503    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:25.696493 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:25.696475    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:26.683447 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:26.683415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:26.778187 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:26.778157    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:26.875733 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:26.875699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:26.976450 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:26.976354    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:27.358682 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:27.358595    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:27.438823 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:27.438770    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:44.803101 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:44.803068    2583 generic.go:356] "Generic (PLEG): container finished" podID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" containerID="1236a90a202e75d47f1eb7d290aa166f67b0e4d680d7646ef49e97a2955945d6" exitCode=0
Jul 28 22:55:44.803523 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:44.803128    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29" event={"ID":"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc","Type":"ContainerDied","Data":"1236a90a202e75d47f1eb7d290aa166f67b0e4d680d7646ef49e97a2955945d6"}
Jul 28 22:55:44.803523 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:44.803497    2583 scope.go:117] "RemoveContainer" containerID="1236a90a202e75d47f1eb7d290aa166f67b0e4d680d7646ef49e97a2955945d6"
Jul 28 22:55:44.804542 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:44.804515    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:55:44.804601 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:44.804586    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:55:44.804638 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:44.804625    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 28 22:55:45.813866 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:45.813834    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29" event={"ID":"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc","Type":"ContainerStarted","Data":"caf646f23b1da518b0947e49022940ca28cd4f0d8d7cfa19f7a1d4568724775c"}
Jul 28 22:55:46.753002 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:46.752966    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:46.753361 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:46.753268    2583 patch_prober.go:28] interesting pod/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 28 22:55:46.753548 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:46.753524    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29" podUID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 28 22:55:47.755212 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:47.753139    2583 patch_prober.go:28] interesting pod/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 28 22:55:47.755212 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:47.753202    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29" podUID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 28 22:55:48.753538 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:48.753489    2583 patch_prober.go:28] interesting pod/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 28 22:55:48.753759 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:48.753560    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29" podUID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 28 22:55:49.753983 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:49.753931    2583 patch_prober.go:28] interesting pod/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Jul 28 22:55:49.754614 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:49.754021    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29" podUID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Jul 28 22:55:50.757479 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:50.757444    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:50.757978 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:50.757827    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:50.758381 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:50.758361    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:55:51.685752 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:51.685717    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:55:56.297750 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:56.297716    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jrfz7"]
Jul 28 22:55:56.301426 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:56.301410    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7"
Jul 28 22:55:56.303297 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:56.303275    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-g4wck\""
Jul 28 22:55:56.308717 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:56.308696    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jrfz7"]
Jul 28 22:55:56.437085 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:56.437041    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mfqb\" (UniqueName: \"kubernetes.io/projected/3b4bad7a-a25e-4084-8c51-26a4f2ac75f8-kube-api-access-9mfqb\") pod \"kuadrant-operator-catalog-jrfz7\" (UID: \"3b4bad7a-a25e-4084-8c51-26a4f2ac75f8\") " pod="kuadrant-system/kuadrant-operator-catalog-jrfz7"
Jul 28 22:55:56.537920 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:56.537886    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mfqb\" (UniqueName: \"kubernetes.io/projected/3b4bad7a-a25e-4084-8c51-26a4f2ac75f8-kube-api-access-9mfqb\") pod \"kuadrant-operator-catalog-jrfz7\" (UID: \"3b4bad7a-a25e-4084-8c51-26a4f2ac75f8\") " pod="kuadrant-system/kuadrant-operator-catalog-jrfz7"
Jul 28 22:55:56.545323 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:56.545298    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mfqb\" (UniqueName: \"kubernetes.io/projected/3b4bad7a-a25e-4084-8c51-26a4f2ac75f8-kube-api-access-9mfqb\") pod \"kuadrant-operator-catalog-jrfz7\" (UID: \"3b4bad7a-a25e-4084-8c51-26a4f2ac75f8\") " pod="kuadrant-system/kuadrant-operator-catalog-jrfz7"
Jul 28 22:55:56.612292 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:56.612209    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7"
Jul 28 22:55:56.744600 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:56.744576    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jrfz7"]
Jul 28 22:55:56.746216 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:55:56.746191    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b4bad7a_a25e_4084_8c51_26a4f2ac75f8.slice/crio-e43953bf162497e3aa1a8bab444c3b2dcfb3f185ba3a4be7c176ba872d02d256 WatchSource:0}: Error finding container e43953bf162497e3aa1a8bab444c3b2dcfb3f185ba3a4be7c176ba872d02d256: Status 404 returned error can't find the container with id e43953bf162497e3aa1a8bab444c3b2dcfb3f185ba3a4be7c176ba872d02d256
Jul 28 22:55:56.897128 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:56.897098    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7" event={"ID":"3b4bad7a-a25e-4084-8c51-26a4f2ac75f8","Type":"ContainerStarted","Data":"e43953bf162497e3aa1a8bab444c3b2dcfb3f185ba3a4be7c176ba872d02d256"}
Jul 28 22:55:59.916864 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:59.916824    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7" event={"ID":"3b4bad7a-a25e-4084-8c51-26a4f2ac75f8","Type":"ContainerStarted","Data":"8f6f64b2123a7d039188c396aa4a6b7987901b155d2e65783d801fe9e622f93d"}
Jul 28 22:55:59.929993 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:55:59.929953    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7" podStartSLOduration=1.7474537350000001 podStartE2EDuration="3.929939581s" totalImagesPullingTime="2.182485846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:55:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:55:59.929863294 +0000 UTC m=+3103.937728879" watchObservedRunningTime="2026-07-28 22:55:59.929939581 +0000 UTC m=+3103.937805138"
Jul 28 22:56:00.620707 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:00.620677    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:56:06.613149 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:06.613104    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7"
Jul 28 22:56:06.613647 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:06.613174    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7"
Jul 28 22:56:06.635151 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:06.635123    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7"
Jul 28 22:56:06.982545 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:06.982519    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7"
Jul 28 22:56:07.015349 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:07.015313    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jrfz7"]
Jul 28 22:56:08.971386 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:08.971324    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7" podUID="3b4bad7a-a25e-4084-8c51-26a4f2ac75f8" containerName="registry-server" containerID="cri-o://8f6f64b2123a7d039188c396aa4a6b7987901b155d2e65783d801fe9e622f93d" gracePeriod=2
Jul 28 22:56:09.224413 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:09.224352    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7"
Jul 28 22:56:09.259310 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:09.259281    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mfqb\" (UniqueName: \"kubernetes.io/projected/3b4bad7a-a25e-4084-8c51-26a4f2ac75f8-kube-api-access-9mfqb\") pod \"3b4bad7a-a25e-4084-8c51-26a4f2ac75f8\" (UID: \"3b4bad7a-a25e-4084-8c51-26a4f2ac75f8\") "
Jul 28 22:56:09.261330 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:09.261305    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b4bad7a-a25e-4084-8c51-26a4f2ac75f8-kube-api-access-9mfqb" (OuterVolumeSpecName: "kube-api-access-9mfqb") pod "3b4bad7a-a25e-4084-8c51-26a4f2ac75f8" (UID: "3b4bad7a-a25e-4084-8c51-26a4f2ac75f8"). InnerVolumeSpecName "kube-api-access-9mfqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:56:09.360692 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:09.360662    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mfqb\" (UniqueName: \"kubernetes.io/projected/3b4bad7a-a25e-4084-8c51-26a4f2ac75f8-kube-api-access-9mfqb\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:56:09.978676 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:09.978639    2583 generic.go:356] "Generic (PLEG): container finished" podID="3b4bad7a-a25e-4084-8c51-26a4f2ac75f8" containerID="8f6f64b2123a7d039188c396aa4a6b7987901b155d2e65783d801fe9e622f93d" exitCode=0
Jul 28 22:56:09.979147 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:09.978683    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7"
Jul 28 22:56:09.979147 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:09.978720    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7" event={"ID":"3b4bad7a-a25e-4084-8c51-26a4f2ac75f8","Type":"ContainerDied","Data":"8f6f64b2123a7d039188c396aa4a6b7987901b155d2e65783d801fe9e622f93d"}
Jul 28 22:56:09.979147 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:09.978778    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jrfz7" event={"ID":"3b4bad7a-a25e-4084-8c51-26a4f2ac75f8","Type":"ContainerDied","Data":"e43953bf162497e3aa1a8bab444c3b2dcfb3f185ba3a4be7c176ba872d02d256"}
Jul 28 22:56:09.979147 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:09.978816    2583 scope.go:117] "RemoveContainer" containerID="8f6f64b2123a7d039188c396aa4a6b7987901b155d2e65783d801fe9e622f93d"
Jul 28 22:56:09.989131 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:09.989108    2583 scope.go:117] "RemoveContainer" containerID="8f6f64b2123a7d039188c396aa4a6b7987901b155d2e65783d801fe9e622f93d"
Jul 28 22:56:09.989370 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:56:09.989347    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f6f64b2123a7d039188c396aa4a6b7987901b155d2e65783d801fe9e622f93d\": container with ID starting with 8f6f64b2123a7d039188c396aa4a6b7987901b155d2e65783d801fe9e622f93d not found: ID does not exist" containerID="8f6f64b2123a7d039188c396aa4a6b7987901b155d2e65783d801fe9e622f93d"
Jul 28 22:56:09.989419 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:09.989378    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f6f64b2123a7d039188c396aa4a6b7987901b155d2e65783d801fe9e622f93d"} err="failed to get container status \"8f6f64b2123a7d039188c396aa4a6b7987901b155d2e65783d801fe9e622f93d\": rpc error: code = NotFound desc = could not find container \"8f6f64b2123a7d039188c396aa4a6b7987901b155d2e65783d801fe9e622f93d\": container with ID starting with 8f6f64b2123a7d039188c396aa4a6b7987901b155d2e65783d801fe9e622f93d not found: ID does not exist"
Jul 28 22:56:10.001871 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:10.001847    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jrfz7"]
Jul 28 22:56:10.005512 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:10.005491    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jrfz7"]
Jul 28 22:56:10.436706 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:10.436673    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b4bad7a-a25e-4084-8c51-26a4f2ac75f8" path="/var/lib/kubelet/pods/3b4bad7a-a25e-4084-8c51-26a4f2ac75f8/volumes"
Jul 28 22:56:49.108475 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:49.108400    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:56:49.433223 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:49.433185    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:56:49.534142 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:49.534107    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:56:52.221525 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.221489    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:56:52.321048 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.321008    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:56:52.408228 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.408181    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"]
Jul 28 22:56:52.409047 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.409023    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b4bad7a-a25e-4084-8c51-26a4f2ac75f8" containerName="registry-server"
Jul 28 22:56:52.409047 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.409048    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b4bad7a-a25e-4084-8c51-26a4f2ac75f8" containerName="registry-server"
Jul 28 22:56:52.409248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.409235    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b4bad7a-a25e-4084-8c51-26a4f2ac75f8" containerName="registry-server"
Jul 28 22:56:52.417451 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.417415    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:56:52.417817 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.417775    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"]
Jul 28 22:56:52.420072 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.420026    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tlhn2\""
Jul 28 22:56:52.420296 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.420280    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-5b3501-serving-cert\""
Jul 28 22:56:52.428500 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.428476    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4\" (UID: \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:56:52.428608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.428512    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqs8k\" (UniqueName: \"kubernetes.io/projected/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-kube-api-access-lqs8k\") pod \"maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4\" (UID: \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:56:52.428608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.428533    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4\" (UID: \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:56:52.438533 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.438503    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:56:52.529843 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.529741    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4\" (UID: \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:56:52.529843 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.529779    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqs8k\" (UniqueName: \"kubernetes.io/projected/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-kube-api-access-lqs8k\") pod \"maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4\" (UID: \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:56:52.529843 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.529811    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4\" (UID: \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:56:52.530142 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:56:52.529922    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-serving-cert: secret "maas-api-e2e-ipp-cleanup-5b3501-serving-cert" not found
Jul 28 22:56:52.530142 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:56:52.530025    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-maas-api-tls podName:4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e nodeName:}" failed. No retries permitted until 2026-07-28 22:56:53.030002577 +0000 UTC m=+3157.037868119 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4" (UID: "4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e") : secret "maas-api-e2e-ipp-cleanup-5b3501-serving-cert" not found
Jul 28 22:56:52.530496 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.530475    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4\" (UID: \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:56:52.539078 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.539059    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqs8k\" (UniqueName: \"kubernetes.io/projected/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-kube-api-access-lqs8k\") pod \"maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4\" (UID: \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:56:52.672882 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:52.672847    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:56:53.034415 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:53.034386    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4\" (UID: \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:56:53.036701 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:53.036678    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4\" (UID: \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:56:53.038866 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:53.038839    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:56:53.131441 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:53.131409    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:56:53.331119 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:53.331039    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:56:53.466205 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:53.466175    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"]
Jul 28 22:56:53.467919 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:56:53.467896    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4807ff4a_7e7e_4a66_972b_5e6c3cf38d2e.slice/crio-d71535cbaf7d3f2a5171fd8ef0d89627f73c0ec75a1d41491409d76031ad446a WatchSource:0}: Error finding container d71535cbaf7d3f2a5171fd8ef0d89627f73c0ec75a1d41491409d76031ad446a: Status 404 returned error can't find the container with id d71535cbaf7d3f2a5171fd8ef0d89627f73c0ec75a1d41491409d76031ad446a
Jul 28 22:56:54.226317 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:54.226280    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4" event={"ID":"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e","Type":"ContainerStarted","Data":"e754d4d4fbb623939395c3cb19575d3796f6485d18939f72790e926afacc0848"}
Jul 28 22:56:54.226499 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:54.226336    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4" event={"ID":"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e","Type":"ContainerStarted","Data":"d71535cbaf7d3f2a5171fd8ef0d89627f73c0ec75a1d41491409d76031ad446a"}
Jul 28 22:56:54.226499 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:54.226357    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:56:54.244814 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:56:54.244756    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4" podStartSLOduration=1.688824301 podStartE2EDuration="2.24474271s" totalImagesPullingTime="555.918409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:56:54.240640136 +0000 UTC m=+3158.248505714" watchObservedRunningTime="2026-07-28 22:56:54.24474271 +0000 UTC m=+3158.252608268"
Jul 28 22:57:00.237677 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:00.237648    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:57:17.266342 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:17.266300    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:17.950300 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:17.950262    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf"]
Jul 28 22:57:17.953844 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:17.953824    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf"
Jul 28 22:57:17.955875 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:17.955855    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:57:17.962296 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:17.961980    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf"]
Jul 28 22:57:18.061955 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:18.061920    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3f2382e-27c1-4763-b91f-df1818bd155c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf\" (UID: \"f3f2382e-27c1-4763-b91f-df1818bd155c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf"
Jul 28 22:57:18.162960 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:18.162927    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3f2382e-27c1-4763-b91f-df1818bd155c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf\" (UID: \"f3f2382e-27c1-4763-b91f-df1818bd155c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf"
Jul 28 22:57:18.163533 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:18.163507    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3f2382e-27c1-4763-b91f-df1818bd155c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf\" (UID: \"f3f2382e-27c1-4763-b91f-df1818bd155c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf"
Jul 28 22:57:18.266505 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:18.266425    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf"
Jul 28 22:57:18.606413 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:18.606383    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf"]
Jul 28 22:57:18.608861 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:57:18.608835    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3f2382e_27c1_4763_b91f_df1818bd155c.slice/crio-2812e4fc9b7d89945b32e3d21f4128f2bc90132a24818de57d02eb939267b35a WatchSource:0}: Error finding container 2812e4fc9b7d89945b32e3d21f4128f2bc90132a24818de57d02eb939267b35a: Status 404 returned error can't find the container with id 2812e4fc9b7d89945b32e3d21f4128f2bc90132a24818de57d02eb939267b35a
Jul 28 22:57:19.373437 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:19.373400    2583 generic.go:356] "Generic (PLEG): container finished" podID="f3f2382e-27c1-4763-b91f-df1818bd155c" containerID="e2a2a3d47833aca80553db5621275eb3bc0517eff7086078c9d4793af406bd00" exitCode=0
Jul 28 22:57:19.373808 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:19.373485    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf" event={"ID":"f3f2382e-27c1-4763-b91f-df1818bd155c","Type":"ContainerDied","Data":"e2a2a3d47833aca80553db5621275eb3bc0517eff7086078c9d4793af406bd00"}
Jul 28 22:57:19.373808 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:19.373537    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf" event={"ID":"f3f2382e-27c1-4763-b91f-df1818bd155c","Type":"ContainerStarted","Data":"2812e4fc9b7d89945b32e3d21f4128f2bc90132a24818de57d02eb939267b35a"}
Jul 28 22:57:20.516374 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:20.516351    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf"
Jul 28 22:57:20.686474 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:20.686389    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3f2382e-27c1-4763-b91f-df1818bd155c-maas-api-service-ca\") pod \"f3f2382e-27c1-4763-b91f-df1818bd155c\" (UID: \"f3f2382e-27c1-4763-b91f-df1818bd155c\") "
Jul 28 22:57:20.686734 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:20.686711    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3f2382e-27c1-4763-b91f-df1818bd155c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f3f2382e-27c1-4763-b91f-df1818bd155c" (UID: "f3f2382e-27c1-4763-b91f-df1818bd155c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:57:20.787991 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:20.787954    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3f2382e-27c1-4763-b91f-df1818bd155c-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:57:21.385766 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:21.385739    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf"
Jul 28 22:57:21.385937 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:21.385741    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf" event={"ID":"f3f2382e-27c1-4763-b91f-df1818bd155c","Type":"ContainerDied","Data":"2812e4fc9b7d89945b32e3d21f4128f2bc90132a24818de57d02eb939267b35a"}
Jul 28 22:57:21.385937 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:21.385868    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2812e4fc9b7d89945b32e3d21f4128f2bc90132a24818de57d02eb939267b35a"
Jul 28 22:57:23.012541 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.012508    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"]
Jul 28 22:57:23.013077 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.012720    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4" podUID="4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e" containerName="maas-api" containerID="cri-o://e754d4d4fbb623939395c3cb19575d3796f6485d18939f72790e926afacc0848" gracePeriod=30
Jul 28 22:57:23.029268 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.029241    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:23.130587 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.130537    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:23.195969 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.195927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:23.216199 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.216155    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf"]
Jul 28 22:57:23.220674 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.220645    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5b3501-v6dsf"]
Jul 28 22:57:23.278433 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.278376    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:57:23.298868 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.298837    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:23.397709 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.397679    2583 generic.go:356] "Generic (PLEG): container finished" podID="4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e" containerID="e754d4d4fbb623939395c3cb19575d3796f6485d18939f72790e926afacc0848" exitCode=0
Jul 28 22:57:23.397907 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.397723    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"
Jul 28 22:57:23.397907 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.397749    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4" event={"ID":"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e","Type":"ContainerDied","Data":"e754d4d4fbb623939395c3cb19575d3796f6485d18939f72790e926afacc0848"}
Jul 28 22:57:23.397907 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.397790    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4" event={"ID":"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e","Type":"ContainerDied","Data":"d71535cbaf7d3f2a5171fd8ef0d89627f73c0ec75a1d41491409d76031ad446a"}
Jul 28 22:57:23.397907 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.397818    2583 scope.go:117] "RemoveContainer" containerID="e754d4d4fbb623939395c3cb19575d3796f6485d18939f72790e926afacc0848"
Jul 28 22:57:23.408575 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.408556    2583 scope.go:117] "RemoveContainer" containerID="e754d4d4fbb623939395c3cb19575d3796f6485d18939f72790e926afacc0848"
Jul 28 22:57:23.408843 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:57:23.408826    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e754d4d4fbb623939395c3cb19575d3796f6485d18939f72790e926afacc0848\": container with ID starting with e754d4d4fbb623939395c3cb19575d3796f6485d18939f72790e926afacc0848 not found: ID does not exist" containerID="e754d4d4fbb623939395c3cb19575d3796f6485d18939f72790e926afacc0848"
Jul 28 22:57:23.408897 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.408851    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e754d4d4fbb623939395c3cb19575d3796f6485d18939f72790e926afacc0848"} err="failed to get container status \"e754d4d4fbb623939395c3cb19575d3796f6485d18939f72790e926afacc0848\": rpc error: code = NotFound desc = could not find container \"e754d4d4fbb623939395c3cb19575d3796f6485d18939f72790e926afacc0848\": container with ID starting with e754d4d4fbb623939395c3cb19575d3796f6485d18939f72790e926afacc0848 not found: ID does not exist"
Jul 28 22:57:23.410096 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.410075    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-maas-api-tls\") pod \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\" (UID: \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\") "
Jul 28 22:57:23.410175 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.410164    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqs8k\" (UniqueName: \"kubernetes.io/projected/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-kube-api-access-lqs8k\") pod \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\" (UID: \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\") "
Jul 28 22:57:23.410262 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.410252    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-openshift-service-ca\") pod \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\" (UID: \"4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e\") "
Jul 28 22:57:23.410601 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.410560    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e" (UID: "4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:57:23.412184 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.412153    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-kube-api-access-lqs8k" (OuterVolumeSpecName: "kube-api-access-lqs8k") pod "4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e" (UID: "4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e"). InnerVolumeSpecName "kube-api-access-lqs8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:57:23.412270 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.412223    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e" (UID: "4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:57:23.511655 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.511621    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqs8k\" (UniqueName: \"kubernetes.io/projected/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-kube-api-access-lqs8k\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:57:23.511655 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.511648    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-openshift-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:57:23.511655 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.511658    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e-maas-api-tls\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:57:23.597626 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.597548    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:23.734492 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.734457    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"]
Jul 28 22:57:23.740802 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:23.740767    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-5b3501-b754f8854-pl9n4"]
Jul 28 22:57:24.432547 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:24.432511    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e" path="/var/lib/kubelet/pods/4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e/volumes"
Jul 28 22:57:24.433041 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:24.433012    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3f2382e-27c1-4763-b91f-df1818bd155c" path="/var/lib/kubelet/pods/f3f2382e-27c1-4763-b91f-df1818bd155c/volumes"
Jul 28 22:57:29.398095 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:29.398056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:29.464690 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:29.464636    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:29.674457 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:29.674363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:29.921416 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:29.921382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:33.357821 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.357778    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5"]
Jul 28 22:57:33.358248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.358146    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3f2382e-27c1-4763-b91f-df1818bd155c" containerName="revoke-keys"
Jul 28 22:57:33.358248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.358158    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3f2382e-27c1-4763-b91f-df1818bd155c" containerName="revoke-keys"
Jul 28 22:57:33.358248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.358177    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e" containerName="maas-api"
Jul 28 22:57:33.358248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.358183    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e" containerName="maas-api"
Jul 28 22:57:33.358248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.358240    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3f2382e-27c1-4763-b91f-df1818bd155c" containerName="revoke-keys"
Jul 28 22:57:33.358248 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.358248    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4807ff4a-7e7e-4a66-972b-5e6c3cf38d2e" containerName="maas-api"
Jul 28 22:57:33.361147 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.361130    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5"
Jul 28 22:57:33.363098 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.363077    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:57:33.367623 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.367597    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5"]
Jul 28 22:57:33.398635 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.398611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55f3ceb0-e3e8-4343-8809-1bfcb689e240-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5\" (UID: \"55f3ceb0-e3e8-4343-8809-1bfcb689e240\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5"
Jul 28 22:57:33.499000 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.498974    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55f3ceb0-e3e8-4343-8809-1bfcb689e240-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5\" (UID: \"55f3ceb0-e3e8-4343-8809-1bfcb689e240\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5"
Jul 28 22:57:33.499554 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.499535    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55f3ceb0-e3e8-4343-8809-1bfcb689e240-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5\" (UID: \"55f3ceb0-e3e8-4343-8809-1bfcb689e240\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5"
Jul 28 22:57:33.672754 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.672725    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5"
Jul 28 22:57:33.803542 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:33.803368    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5"]
Jul 28 22:57:33.806476 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:57:33.806437    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55f3ceb0_e3e8_4343_8809_1bfcb689e240.slice/crio-4c681a27d1207563b2400b677854081aa4b0f78c36e44c76b6ce1035710aec4e WatchSource:0}: Error finding container 4c681a27d1207563b2400b677854081aa4b0f78c36e44c76b6ce1035710aec4e: Status 404 returned error can't find the container with id 4c681a27d1207563b2400b677854081aa4b0f78c36e44c76b6ce1035710aec4e
Jul 28 22:57:34.461730 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:34.461702    2583 generic.go:356] "Generic (PLEG): container finished" podID="55f3ceb0-e3e8-4343-8809-1bfcb689e240" containerID="76a8a4ad1923670f9b07b8409d2b1cc523c741ed3ed0c2aa959e3243214c6a84" exitCode=0
Jul 28 22:57:34.462055 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:34.461771    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5" event={"ID":"55f3ceb0-e3e8-4343-8809-1bfcb689e240","Type":"ContainerDied","Data":"76a8a4ad1923670f9b07b8409d2b1cc523c741ed3ed0c2aa959e3243214c6a84"}
Jul 28 22:57:34.462055 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:34.461830    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5" event={"ID":"55f3ceb0-e3e8-4343-8809-1bfcb689e240","Type":"ContainerStarted","Data":"4c681a27d1207563b2400b677854081aa4b0f78c36e44c76b6ce1035710aec4e"}
Jul 28 22:57:35.598151 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:35.598128    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5"
Jul 28 22:57:35.617032 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:35.617005    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55f3ceb0-e3e8-4343-8809-1bfcb689e240-maas-api-service-ca\") pod \"55f3ceb0-e3e8-4343-8809-1bfcb689e240\" (UID: \"55f3ceb0-e3e8-4343-8809-1bfcb689e240\") "
Jul 28 22:57:35.617344 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:35.617323    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55f3ceb0-e3e8-4343-8809-1bfcb689e240-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "55f3ceb0-e3e8-4343-8809-1bfcb689e240" (UID: "55f3ceb0-e3e8-4343-8809-1bfcb689e240"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:57:35.718553 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:35.718474    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55f3ceb0-e3e8-4343-8809-1bfcb689e240-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:57:36.473580 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:36.473556    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5"
Jul 28 22:57:36.473580 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:36.473569    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5" event={"ID":"55f3ceb0-e3e8-4343-8809-1bfcb689e240","Type":"ContainerDied","Data":"4c681a27d1207563b2400b677854081aa4b0f78c36e44c76b6ce1035710aec4e"}
Jul 28 22:57:36.473769 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:36.473621    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c681a27d1207563b2400b677854081aa4b0f78c36e44c76b6ce1035710aec4e"
Jul 28 22:57:48.855669 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:48.855635    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:48.930246 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:48.930194    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:48.992909 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:48.992877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:49.032102 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:49.032068    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5"]
Jul 28 22:57:49.034387 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:49.034363    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ff4fa1-wc5w5"]
Jul 28 22:57:49.051932 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:49.051834    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:49.434028 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:49.433994    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:57:50.432745 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:50.432712    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55f3ceb0-e3e8-4343-8809-1bfcb689e240" path="/var/lib/kubelet/pods/55f3ceb0-e3e8-4343-8809-1bfcb689e240/volumes"
Jul 28 22:57:54.511139 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:57:54.511103    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:31.530407 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:31.530368    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:32.833172 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:32.833138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:37.941444 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:37.941406    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:38.013948 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.013914    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:38.245535 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.245447    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"]
Jul 28 22:58:38.246050 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.246018    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" containerName="istio-proxy" containerID="cri-o://b80ff847bbd90bd96ad4f92c2ba1ecf038b8dad929eafc8a95e44f8026d42eaa" gracePeriod=30
Jul 28 22:58:38.268017 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.267984    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:38.540137 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.540036    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:38.667073 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.667039    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9"]
Jul 28 22:58:38.667414 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.667403    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55f3ceb0-e3e8-4343-8809-1bfcb689e240" containerName="revoke-keys"
Jul 28 22:58:38.667459 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.667416    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="55f3ceb0-e3e8-4343-8809-1bfcb689e240" containerName="revoke-keys"
Jul 28 22:58:38.667494 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.667485    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="55f3ceb0-e3e8-4343-8809-1bfcb689e240" containerName="revoke-keys"
Jul 28 22:58:38.670631 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.670616    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9"
Jul 28 22:58:38.672437 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.672416    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:58:38.678539 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.678512    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9"]
Jul 28 22:58:38.787154 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.787127    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40551423-7d29-49d5-be64-833a7d055384-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9\" (UID: \"40551423-7d29-49d5-be64-833a7d055384\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9"
Jul 28 22:58:38.888520 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.888495    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40551423-7d29-49d5-be64-833a7d055384-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9\" (UID: \"40551423-7d29-49d5-be64-833a7d055384\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9"
Jul 28 22:58:38.889115 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.889097    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40551423-7d29-49d5-be64-833a7d055384-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9\" (UID: \"40551423-7d29-49d5-be64-833a7d055384\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9"
Jul 28 22:58:38.981525 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:38.981496    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9"
Jul 28 22:58:39.119475 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:39.119450    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9"]
Jul 28 22:58:39.120867 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:58:39.120833    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40551423_7d29_49d5_be64_833a7d055384.slice/crio-3addc1f0613f167c5e532b16e3bb477c8a69f8ed1ef39a54f0f159e3e809c115 WatchSource:0}: Error finding container 3addc1f0613f167c5e532b16e3bb477c8a69f8ed1ef39a54f0f159e3e809c115: Status 404 returned error can't find the container with id 3addc1f0613f167c5e532b16e3bb477c8a69f8ed1ef39a54f0f159e3e809c115
Jul 28 22:58:39.831029 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:39.830998    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9" event={"ID":"40551423-7d29-49d5-be64-833a7d055384","Type":"ContainerStarted","Data":"94c3e211311fd6422df43c62af2a5f1718274e11359e9266005ac11f0882f8b8"}
Jul 28 22:58:39.831214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:39.831047    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9" event={"ID":"40551423-7d29-49d5-be64-833a7d055384","Type":"ContainerStarted","Data":"3addc1f0613f167c5e532b16e3bb477c8a69f8ed1ef39a54f0f159e3e809c115"}
Jul 28 22:58:39.848937 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:39.848895    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9" podStartSLOduration=1.848881387 podStartE2EDuration="1.848881387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:58:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:58:39.846519374 +0000 UTC m=+3263.854384943" watchObservedRunningTime="2026-07-28 22:58:39.848881387 +0000 UTC m=+3263.856746944"
Jul 28 22:58:40.838161 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:40.838124    2583 generic.go:356] "Generic (PLEG): container finished" podID="40551423-7d29-49d5-be64-833a7d055384" containerID="94c3e211311fd6422df43c62af2a5f1718274e11359e9266005ac11f0882f8b8" exitCode=0
Jul 28 22:58:40.838523 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:40.838194    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9" event={"ID":"40551423-7d29-49d5-be64-833a7d055384","Type":"ContainerDied","Data":"94c3e211311fd6422df43c62af2a5f1718274e11359e9266005ac11f0882f8b8"}
Jul 28 22:58:41.972652 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:41.972632    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9"
Jul 28 22:58:42.017296 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:42.017261    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40551423-7d29-49d5-be64-833a7d055384-maas-api-service-ca\") pod \"40551423-7d29-49d5-be64-833a7d055384\" (UID: \"40551423-7d29-49d5-be64-833a7d055384\") "
Jul 28 22:58:42.017565 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:42.017537    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40551423-7d29-49d5-be64-833a7d055384-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "40551423-7d29-49d5-be64-833a7d055384" (UID: "40551423-7d29-49d5-be64-833a7d055384"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:58:42.118214 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:42.118189    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40551423-7d29-49d5-be64-833a7d055384-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:58:42.851161 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:42.851072    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9"
Jul 28 22:58:42.851161 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:42.851112    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9" event={"ID":"40551423-7d29-49d5-be64-833a7d055384","Type":"ContainerDied","Data":"3addc1f0613f167c5e532b16e3bb477c8a69f8ed1ef39a54f0f159e3e809c115"}
Jul 28 22:58:42.851161 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:42.851157    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3addc1f0613f167c5e532b16e3bb477c8a69f8ed1ef39a54f0f159e3e809c115"
Jul 28 22:58:43.494514 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.494493    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:58:43.632891 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.632864    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-workload-socket\") pod \"cc664d93-a103-4deb-b939-d312a1f435a1\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") "
Jul 28 22:58:43.633025 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.632900    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-credential-socket\") pod \"cc664d93-a103-4deb-b939-d312a1f435a1\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") "
Jul 28 22:58:43.633025 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.632922    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc664d93-a103-4deb-b939-d312a1f435a1-istio-podinfo\") pod \"cc664d93-a103-4deb-b939-d312a1f435a1\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") "
Jul 28 22:58:43.633025 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.632970    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-workload-certs\") pod \"cc664d93-a103-4deb-b939-d312a1f435a1\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") "
Jul 28 22:58:43.633150 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.633040    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-istio-envoy\") pod \"cc664d93-a103-4deb-b939-d312a1f435a1\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") "
Jul 28 22:58:43.633150 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.633068    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76kq9\" (UniqueName: \"kubernetes.io/projected/cc664d93-a103-4deb-b939-d312a1f435a1-kube-api-access-76kq9\") pod \"cc664d93-a103-4deb-b939-d312a1f435a1\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") "
Jul 28 22:58:43.633150 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.633117    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-istio-data\") pod \"cc664d93-a103-4deb-b939-d312a1f435a1\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") "
Jul 28 22:58:43.633150 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.633143    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc664d93-a103-4deb-b939-d312a1f435a1-istio-token\") pod \"cc664d93-a103-4deb-b939-d312a1f435a1\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") "
Jul 28 22:58:43.633359 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.633151    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cc664d93-a103-4deb-b939-d312a1f435a1" (UID: "cc664d93-a103-4deb-b939-d312a1f435a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:58:43.633359 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.633188    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc664d93-a103-4deb-b939-d312a1f435a1-istiod-ca-cert\") pod \"cc664d93-a103-4deb-b939-d312a1f435a1\" (UID: \"cc664d93-a103-4deb-b939-d312a1f435a1\") "
Jul 28 22:58:43.633359 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.633207    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cc664d93-a103-4deb-b939-d312a1f435a1" (UID: "cc664d93-a103-4deb-b939-d312a1f435a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:58:43.633359 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.633313    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cc664d93-a103-4deb-b939-d312a1f435a1" (UID: "cc664d93-a103-4deb-b939-d312a1f435a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:58:43.633598 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.633513    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc664d93-a103-4deb-b939-d312a1f435a1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cc664d93-a103-4deb-b939-d312a1f435a1" (UID: "cc664d93-a103-4deb-b939-d312a1f435a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:58:43.633598 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.633537    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-workload-socket\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:58:43.633598 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.633556    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-credential-socket\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:58:43.633598 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.633569    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-workload-certs\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:58:43.634226 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.634198    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-istio-data" (OuterVolumeSpecName: "istio-data") pod "cc664d93-a103-4deb-b939-d312a1f435a1" (UID: "cc664d93-a103-4deb-b939-d312a1f435a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:58:43.635274 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.635244    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cc664d93-a103-4deb-b939-d312a1f435a1" (UID: "cc664d93-a103-4deb-b939-d312a1f435a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:58:43.635372 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.635288    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc664d93-a103-4deb-b939-d312a1f435a1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cc664d93-a103-4deb-b939-d312a1f435a1" (UID: "cc664d93-a103-4deb-b939-d312a1f435a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:58:43.635372 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.635315    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc664d93-a103-4deb-b939-d312a1f435a1-kube-api-access-76kq9" (OuterVolumeSpecName: "kube-api-access-76kq9") pod "cc664d93-a103-4deb-b939-d312a1f435a1" (UID: "cc664d93-a103-4deb-b939-d312a1f435a1"). InnerVolumeSpecName "kube-api-access-76kq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:58:43.635448 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.635379    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc664d93-a103-4deb-b939-d312a1f435a1-istio-token" (OuterVolumeSpecName: "istio-token") pod "cc664d93-a103-4deb-b939-d312a1f435a1" (UID: "cc664d93-a103-4deb-b939-d312a1f435a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:58:43.738242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.734493    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc664d93-a103-4deb-b939-d312a1f435a1-istio-podinfo\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:58:43.738242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.734525    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-istio-envoy\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:58:43.738242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.734539    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76kq9\" (UniqueName: \"kubernetes.io/projected/cc664d93-a103-4deb-b939-d312a1f435a1-kube-api-access-76kq9\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:58:43.738242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.734561    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc664d93-a103-4deb-b939-d312a1f435a1-istio-data\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:58:43.738242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.734576    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc664d93-a103-4deb-b939-d312a1f435a1-istio-token\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:58:43.738242 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.734591    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc664d93-a103-4deb-b939-d312a1f435a1-istiod-ca-cert\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:58:43.858429 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.858391    2583 generic.go:356] "Generic (PLEG): container finished" podID="cc664d93-a103-4deb-b939-d312a1f435a1" containerID="b80ff847bbd90bd96ad4f92c2ba1ecf038b8dad929eafc8a95e44f8026d42eaa" exitCode=0
Jul 28 22:58:43.858608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.858459    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"
Jul 28 22:58:43.858608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.858479    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" event={"ID":"cc664d93-a103-4deb-b939-d312a1f435a1","Type":"ContainerDied","Data":"b80ff847bbd90bd96ad4f92c2ba1ecf038b8dad929eafc8a95e44f8026d42eaa"}
Jul 28 22:58:43.858608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.858563    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8" event={"ID":"cc664d93-a103-4deb-b939-d312a1f435a1","Type":"ContainerDied","Data":"d74584b48a5e54d2e9443d139ba8933b45be9f044e24abb73022e58cdda5568b"}
Jul 28 22:58:43.858608 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.858588    2583 scope.go:117] "RemoveContainer" containerID="b80ff847bbd90bd96ad4f92c2ba1ecf038b8dad929eafc8a95e44f8026d42eaa"
Jul 28 22:58:43.868550 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.868532    2583 scope.go:117] "RemoveContainer" containerID="4c3ad29bb0b9b23233b2c09cd2dcef9bbb1a5232aa7d3d2fb19a1a288ef076d1"
Jul 28 22:58:43.876846 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.876733    2583 scope.go:117] "RemoveContainer" containerID="b80ff847bbd90bd96ad4f92c2ba1ecf038b8dad929eafc8a95e44f8026d42eaa"
Jul 28 22:58:43.877196 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:58:43.877170    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b80ff847bbd90bd96ad4f92c2ba1ecf038b8dad929eafc8a95e44f8026d42eaa\": container with ID starting with b80ff847bbd90bd96ad4f92c2ba1ecf038b8dad929eafc8a95e44f8026d42eaa not found: ID does not exist" containerID="b80ff847bbd90bd96ad4f92c2ba1ecf038b8dad929eafc8a95e44f8026d42eaa"
Jul 28 22:58:43.877298 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.877205    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b80ff847bbd90bd96ad4f92c2ba1ecf038b8dad929eafc8a95e44f8026d42eaa"} err="failed to get container status \"b80ff847bbd90bd96ad4f92c2ba1ecf038b8dad929eafc8a95e44f8026d42eaa\": rpc error: code = NotFound desc = could not find container \"b80ff847bbd90bd96ad4f92c2ba1ecf038b8dad929eafc8a95e44f8026d42eaa\": container with ID starting with b80ff847bbd90bd96ad4f92c2ba1ecf038b8dad929eafc8a95e44f8026d42eaa not found: ID does not exist"
Jul 28 22:58:43.877298 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.877227    2583 scope.go:117] "RemoveContainer" containerID="4c3ad29bb0b9b23233b2c09cd2dcef9bbb1a5232aa7d3d2fb19a1a288ef076d1"
Jul 28 22:58:43.877502 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:58:43.877477    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c3ad29bb0b9b23233b2c09cd2dcef9bbb1a5232aa7d3d2fb19a1a288ef076d1\": container with ID starting with 4c3ad29bb0b9b23233b2c09cd2dcef9bbb1a5232aa7d3d2fb19a1a288ef076d1 not found: ID does not exist" containerID="4c3ad29bb0b9b23233b2c09cd2dcef9bbb1a5232aa7d3d2fb19a1a288ef076d1"
Jul 28 22:58:43.877601 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.877508    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c3ad29bb0b9b23233b2c09cd2dcef9bbb1a5232aa7d3d2fb19a1a288ef076d1"} err="failed to get container status \"4c3ad29bb0b9b23233b2c09cd2dcef9bbb1a5232aa7d3d2fb19a1a288ef076d1\": rpc error: code = NotFound desc = could not find container \"4c3ad29bb0b9b23233b2c09cd2dcef9bbb1a5232aa7d3d2fb19a1a288ef076d1\": container with ID starting with 4c3ad29bb0b9b23233b2c09cd2dcef9bbb1a5232aa7d3d2fb19a1a288ef076d1 not found: ID does not exist"
Jul 28 22:58:43.879454 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.879436    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"]
Jul 28 22:58:43.883871 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:43.883821    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-ff4fa1-openshift-default-7c57498d7d-jnmr8"]
Jul 28 22:58:44.432366 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:44.432331    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" path="/var/lib/kubelet/pods/cc664d93-a103-4deb-b939-d312a1f435a1/volumes"
Jul 28 22:58:49.360896 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:49.360856    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:49.477989 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:49.477950    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:49.576324 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:49.576283    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:49.649336 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:49.649302    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9"]
Jul 28 22:58:49.651635 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:49.651608    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:49.652385 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:49.652364    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-056c8a-cgnb9"]
Jul 28 22:58:49.708363 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:49.708334    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:50.028601 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:50.028511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:50.431175 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:50.431146    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40551423-7d29-49d5-be64-833a7d055384" path="/var/lib/kubelet/pods/40551423-7d29-49d5-be64-833a7d055384/volumes"
Jul 28 22:58:55.945781 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:55.945749    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"]
Jul 28 22:58:55.946213 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:55.945978    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29" podUID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" containerName="istio-proxy" containerID="cri-o://caf646f23b1da518b0947e49022940ca28cd4f0d8d7cfa19f7a1d4568724775c" gracePeriod=30
Jul 28 22:58:56.338411 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.338329    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk"]
Jul 28 22:58:56.338723 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.338711    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" containerName="istio-proxy"
Jul 28 22:58:56.338780 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.338725    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" containerName="istio-proxy"
Jul 28 22:58:56.338780 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.338749    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40551423-7d29-49d5-be64-833a7d055384" containerName="revoke-keys"
Jul 28 22:58:56.338780 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.338755    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="40551423-7d29-49d5-be64-833a7d055384" containerName="revoke-keys"
Jul 28 22:58:56.338909 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.338838    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" containerName="istio-proxy"
Jul 28 22:58:56.338909 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.338846    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="40551423-7d29-49d5-be64-833a7d055384" containerName="revoke-keys"
Jul 28 22:58:56.338909 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.338853    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" containerName="istio-proxy"
Jul 28 22:58:56.343513 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.343496    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk"
Jul 28 22:58:56.345648 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.345621    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:58:56.348978 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.348958    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk"]
Jul 28 22:58:56.452873 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.452840    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/833cfd29-8267-42e6-a0b9-67651cc5e92b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk\" (UID: \"833cfd29-8267-42e6-a0b9-67651cc5e92b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk"
Jul 28 22:58:56.553658 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.553622    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/833cfd29-8267-42e6-a0b9-67651cc5e92b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk\" (UID: \"833cfd29-8267-42e6-a0b9-67651cc5e92b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk"
Jul 28 22:58:56.554259 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.554240    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/833cfd29-8267-42e6-a0b9-67651cc5e92b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk\" (UID: \"833cfd29-8267-42e6-a0b9-67651cc5e92b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk"
Jul 28 22:58:56.558734 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.558705    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:56.656153 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.656128    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk"
Jul 28 22:58:56.659209 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.659001    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:58:56.791049 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.791023    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk"]
Jul 28 22:58:56.793569 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:58:56.793534    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod833cfd29_8267_42e6_a0b9_67651cc5e92b.slice/crio-f11349cba3eb87e465d942d16fb753da70b0ced8021fdcf06e2c10fe6f563746 WatchSource:0}: Error finding container f11349cba3eb87e465d942d16fb753da70b0ced8021fdcf06e2c10fe6f563746: Status 404 returned error can't find the container with id f11349cba3eb87e465d942d16fb753da70b0ced8021fdcf06e2c10fe6f563746
Jul 28 22:58:56.933968 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:56.933888    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk" event={"ID":"833cfd29-8267-42e6-a0b9-67651cc5e92b","Type":"ContainerStarted","Data":"f11349cba3eb87e465d942d16fb753da70b0ced8021fdcf06e2c10fe6f563746"}
Jul 28 22:58:57.940704 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:57.940676    2583 generic.go:356] "Generic (PLEG): container finished" podID="833cfd29-8267-42e6-a0b9-67651cc5e92b" containerID="86da947f0b27d11f222ad274527852feeb0cf517a763a2a76d80e63f05679c49" exitCode=0
Jul 28 22:58:57.941098 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:57.940773    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk" event={"ID":"833cfd29-8267-42e6-a0b9-67651cc5e92b","Type":"ContainerDied","Data":"86da947f0b27d11f222ad274527852feeb0cf517a763a2a76d80e63f05679c49"}
Jul 28 22:58:59.083229 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:59.083204    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk"
Jul 28 22:58:59.175927 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:59.175895    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/833cfd29-8267-42e6-a0b9-67651cc5e92b-maas-api-service-ca\") pod \"833cfd29-8267-42e6-a0b9-67651cc5e92b\" (UID: \"833cfd29-8267-42e6-a0b9-67651cc5e92b\") "
Jul 28 22:58:59.176270 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:59.176250    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/833cfd29-8267-42e6-a0b9-67651cc5e92b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "833cfd29-8267-42e6-a0b9-67651cc5e92b" (UID: "833cfd29-8267-42e6-a0b9-67651cc5e92b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:58:59.277556 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:59.277487    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/833cfd29-8267-42e6-a0b9-67651cc5e92b-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:58:59.955584 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:59.955554    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk"
Jul 28 22:58:59.955584 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:59.955577    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk" event={"ID":"833cfd29-8267-42e6-a0b9-67651cc5e92b","Type":"ContainerDied","Data":"f11349cba3eb87e465d942d16fb753da70b0ced8021fdcf06e2c10fe6f563746"}
Jul 28 22:58:59.955790 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:58:59.955623    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f11349cba3eb87e465d942d16fb753da70b0ced8021fdcf06e2c10fe6f563746"
Jul 28 22:59:01.210551 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.210494    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:59:01.297558 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.297519    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-data\") pod \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") "
Jul 28 22:59:01.297735 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.297579    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istiod-ca-cert\") pod \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") "
Jul 28 22:59:01.297735 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.297648    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-token\") pod \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") "
Jul 28 22:59:01.297735 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.297678    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-workload-socket\") pod \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") "
Jul 28 22:59:01.297735 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.297707    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-envoy\") pod \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") "
Jul 28 22:59:01.297970 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.297771    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fv7nr\" (UniqueName: \"kubernetes.io/projected/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-kube-api-access-fv7nr\") pod \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") "
Jul 28 22:59:01.297970 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.297835    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-podinfo\") pod \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") "
Jul 28 22:59:01.297970 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.297905    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-credential-socket\") pod \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") "
Jul 28 22:59:01.297970 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.297929    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-workload-certs\") pod \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\" (UID: \"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc\") "
Jul 28 22:59:01.298182 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.298083    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" (UID: "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:59:01.298182 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.298117    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" (UID: "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:01.298310 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.298286    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" (UID: "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:01.298403 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.298377    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-workload-certs\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:01.298468 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.298403    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istiod-ca-cert\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:01.298468 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.298417    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-workload-socket\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:01.298575 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.298506    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" (UID: "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:01.298633 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.298610    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-data" (OuterVolumeSpecName: "istio-data") pod "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" (UID: "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:01.300095 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.300067    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-token" (OuterVolumeSpecName: "istio-token") pod "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" (UID: "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:59:01.300095 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.300080    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" (UID: "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:01.300369 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.300348    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-kube-api-access-fv7nr" (OuterVolumeSpecName: "kube-api-access-fv7nr") pod "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" (UID: "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc"). InnerVolumeSpecName "kube-api-access-fv7nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:59:01.300437 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.300408    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" (UID: "aea64ae1-5d5a-44e7-8f8f-4c397c7533dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:59:01.399789 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.399755    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fv7nr\" (UniqueName: \"kubernetes.io/projected/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-kube-api-access-fv7nr\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:01.399789 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.399784    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-podinfo\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:01.400011 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.399821    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-credential-socket\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:01.400011 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.399830    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-data\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:01.400011 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.399840    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-token\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:01.400011 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.399848    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc-istio-envoy\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:01.592953 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.592864    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:01.969120 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.969087    2583 generic.go:356] "Generic (PLEG): container finished" podID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" containerID="caf646f23b1da518b0947e49022940ca28cd4f0d8d7cfa19f7a1d4568724775c" exitCode=0
Jul 28 22:59:01.969304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.969154    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29" event={"ID":"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc","Type":"ContainerDied","Data":"caf646f23b1da518b0947e49022940ca28cd4f0d8d7cfa19f7a1d4568724775c"}
Jul 28 22:59:01.969304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.969161    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"
Jul 28 22:59:01.969304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.969209    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29" event={"ID":"aea64ae1-5d5a-44e7-8f8f-4c397c7533dc","Type":"ContainerDied","Data":"0d9fb46baf9e017c0ea587bc9821201c0fe3a1206c5a9c0d4ab83cf88da6d731"}
Jul 28 22:59:01.969304 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.969240    2583 scope.go:117] "RemoveContainer" containerID="caf646f23b1da518b0947e49022940ca28cd4f0d8d7cfa19f7a1d4568724775c"
Jul 28 22:59:01.979629 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.979610    2583 scope.go:117] "RemoveContainer" containerID="1236a90a202e75d47f1eb7d290aa166f67b0e4d680d7646ef49e97a2955945d6"
Jul 28 22:59:01.987714 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.987693    2583 scope.go:117] "RemoveContainer" containerID="caf646f23b1da518b0947e49022940ca28cd4f0d8d7cfa19f7a1d4568724775c"
Jul 28 22:59:01.987963 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:59:01.987944    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caf646f23b1da518b0947e49022940ca28cd4f0d8d7cfa19f7a1d4568724775c\": container with ID starting with caf646f23b1da518b0947e49022940ca28cd4f0d8d7cfa19f7a1d4568724775c not found: ID does not exist" containerID="caf646f23b1da518b0947e49022940ca28cd4f0d8d7cfa19f7a1d4568724775c"
Jul 28 22:59:01.988034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.987971    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caf646f23b1da518b0947e49022940ca28cd4f0d8d7cfa19f7a1d4568724775c"} err="failed to get container status \"caf646f23b1da518b0947e49022940ca28cd4f0d8d7cfa19f7a1d4568724775c\": rpc error: code = NotFound desc = could not find container \"caf646f23b1da518b0947e49022940ca28cd4f0d8d7cfa19f7a1d4568724775c\": container with ID starting with caf646f23b1da518b0947e49022940ca28cd4f0d8d7cfa19f7a1d4568724775c not found: ID does not exist"
Jul 28 22:59:01.988034 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.987989    2583 scope.go:117] "RemoveContainer" containerID="1236a90a202e75d47f1eb7d290aa166f67b0e4d680d7646ef49e97a2955945d6"
Jul 28 22:59:01.988214 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:59:01.988198    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1236a90a202e75d47f1eb7d290aa166f67b0e4d680d7646ef49e97a2955945d6\": container with ID starting with 1236a90a202e75d47f1eb7d290aa166f67b0e4d680d7646ef49e97a2955945d6 not found: ID does not exist" containerID="1236a90a202e75d47f1eb7d290aa166f67b0e4d680d7646ef49e97a2955945d6"
Jul 28 22:59:01.988258 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.988217    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1236a90a202e75d47f1eb7d290aa166f67b0e4d680d7646ef49e97a2955945d6"} err="failed to get container status \"1236a90a202e75d47f1eb7d290aa166f67b0e4d680d7646ef49e97a2955945d6\": rpc error: code = NotFound desc = could not find container \"1236a90a202e75d47f1eb7d290aa166f67b0e4d680d7646ef49e97a2955945d6\": container with ID starting with 1236a90a202e75d47f1eb7d290aa166f67b0e4d680d7646ef49e97a2955945d6 not found: ID does not exist"
Jul 28 22:59:01.991199 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.991179    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"]
Jul 28 22:59:01.994696 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:01.994678    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-056c8a-openshift-default-77b6bb7d78-rbh29"]
Jul 28 22:59:02.431505 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:02.431474    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" path="/var/lib/kubelet/pods/aea64ae1-5d5a-44e7-8f8f-4c397c7533dc/volumes"
Jul 28 22:59:07.043441 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:07.043405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:07.130222 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:07.130192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:07.202969 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:07.202930    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:07.254453 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:07.254422    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk"]
Jul 28 22:59:07.258409 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:07.258387    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ae4fa9-q9jxk"]
Jul 28 22:59:07.282707 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:07.282674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:07.640454 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:07.640420    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:08.431480 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:08.431448    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="833cfd29-8267-42e6-a0b9-67651cc5e92b" path="/var/lib/kubelet/pods/833cfd29-8267-42e6-a0b9-67651cc5e92b/volumes"
Jul 28 22:59:13.610194 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:13.610163    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:13.713070 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:13.713024    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:13.857433 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:13.857403    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:14.106920 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.106886    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:14.225159 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.225121    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc"]
Jul 28 22:59:14.225570 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.225554    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" containerName="istio-proxy"
Jul 28 22:59:14.225652 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.225574    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc664d93-a103-4deb-b939-d312a1f435a1" containerName="istio-proxy"
Jul 28 22:59:14.225652 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.225597    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" containerName="istio-proxy"
Jul 28 22:59:14.225652 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.225605    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" containerName="istio-proxy"
Jul 28 22:59:14.225652 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.225622    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="833cfd29-8267-42e6-a0b9-67651cc5e92b" containerName="revoke-keys"
Jul 28 22:59:14.225652 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.225632    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="833cfd29-8267-42e6-a0b9-67651cc5e92b" containerName="revoke-keys"
Jul 28 22:59:14.225946 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.225656    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" containerName="istio-proxy"
Jul 28 22:59:14.225946 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.225664    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" containerName="istio-proxy"
Jul 28 22:59:14.225946 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.225753    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" containerName="istio-proxy"
Jul 28 22:59:14.225946 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.225767    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="aea64ae1-5d5a-44e7-8f8f-4c397c7533dc" containerName="istio-proxy"
Jul 28 22:59:14.225946 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.225779    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="833cfd29-8267-42e6-a0b9-67651cc5e92b" containerName="revoke-keys"
Jul 28 22:59:14.230388 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.230369    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc"
Jul 28 22:59:14.232360 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.232340    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6zfvv\""
Jul 28 22:59:14.236140 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.236118    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc"]
Jul 28 22:59:14.313821 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.313765    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5da9eada-18c2-4c38-a28a-e52262a5c20a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc\" (UID: \"5da9eada-18c2-4c38-a28a-e52262a5c20a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc"
Jul 28 22:59:14.415045 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.414996    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5da9eada-18c2-4c38-a28a-e52262a5c20a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc\" (UID: \"5da9eada-18c2-4c38-a28a-e52262a5c20a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc"
Jul 28 22:59:14.415642 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.415620    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5da9eada-18c2-4c38-a28a-e52262a5c20a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc\" (UID: \"5da9eada-18c2-4c38-a28a-e52262a5c20a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc"
Jul 28 22:59:14.541780 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.541747    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc"
Jul 28 22:59:14.676354 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:14.676324    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc"]
Jul 28 22:59:14.678103 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:59:14.678070    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da9eada_18c2_4c38_a28a_e52262a5c20a.slice/crio-f6860d1fc57417b5315496c4716e2dbe7fca149d481ef7099d9639fc154c7ac0 WatchSource:0}: Error finding container f6860d1fc57417b5315496c4716e2dbe7fca149d481ef7099d9639fc154c7ac0: Status 404 returned error can't find the container with id f6860d1fc57417b5315496c4716e2dbe7fca149d481ef7099d9639fc154c7ac0
Jul 28 22:59:15.046215 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:15.046175    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc" event={"ID":"5da9eada-18c2-4c38-a28a-e52262a5c20a","Type":"ContainerStarted","Data":"86b0f65fe369e3c17bbd73d2d1ed7eb6f68353d16c2e772203290c51220d9315"}
Jul 28 22:59:15.046383 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:15.046275    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc" event={"ID":"5da9eada-18c2-4c38-a28a-e52262a5c20a","Type":"ContainerStarted","Data":"f6860d1fc57417b5315496c4716e2dbe7fca149d481ef7099d9639fc154c7ac0"}
Jul 28 22:59:15.074528 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:15.074475    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc" podStartSLOduration=1.074459089 podStartE2EDuration="1.074459089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:59:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:59:15.072205296 +0000 UTC m=+3299.080070866" watchObservedRunningTime="2026-07-28 22:59:15.074459089 +0000 UTC m=+3299.082324647"
Jul 28 22:59:16.053314 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:16.053280    2583 generic.go:356] "Generic (PLEG): container finished" podID="5da9eada-18c2-4c38-a28a-e52262a5c20a" containerID="86b0f65fe369e3c17bbd73d2d1ed7eb6f68353d16c2e772203290c51220d9315" exitCode=0
Jul 28 22:59:16.053740 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:16.053373    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc" event={"ID":"5da9eada-18c2-4c38-a28a-e52262a5c20a","Type":"ContainerDied","Data":"86b0f65fe369e3c17bbd73d2d1ed7eb6f68353d16c2e772203290c51220d9315"}
Jul 28 22:59:16.692011 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:16.691928    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:59:16.717467 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:16.717440    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 22:59:17.189228 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:17.189205    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc"
Jul 28 22:59:17.241110 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:17.241075    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5da9eada-18c2-4c38-a28a-e52262a5c20a-maas-api-service-ca\") pod \"5da9eada-18c2-4c38-a28a-e52262a5c20a\" (UID: \"5da9eada-18c2-4c38-a28a-e52262a5c20a\") "
Jul 28 22:59:17.241422 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:17.241400    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5da9eada-18c2-4c38-a28a-e52262a5c20a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5da9eada-18c2-4c38-a28a-e52262a5c20a" (UID: "5da9eada-18c2-4c38-a28a-e52262a5c20a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:59:17.342279 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:17.342182    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5da9eada-18c2-4c38-a28a-e52262a5c20a-maas-api-service-ca\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:18.066187 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:18.066157    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc"
Jul 28 22:59:18.066364 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:18.066188    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc" event={"ID":"5da9eada-18c2-4c38-a28a-e52262a5c20a","Type":"ContainerDied","Data":"f6860d1fc57417b5315496c4716e2dbe7fca149d481ef7099d9639fc154c7ac0"}
Jul 28 22:59:18.066364 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:18.066230    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6860d1fc57417b5315496c4716e2dbe7fca149d481ef7099d9639fc154c7ac0"
Jul 28 22:59:24.681867 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:24.681835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:24.768634 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:24.768597    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:24.833203 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:24.833162    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:24.866279 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:24.866248    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc"]
Jul 28 22:59:24.868354 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:24.868334    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1661d9-r4ggc"]
Jul 28 22:59:24.905020 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:24.904982    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:25.264904 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:25.264871    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:26.431629 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:26.431596    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5da9eada-18c2-4c38-a28a-e52262a5c20a" path="/var/lib/kubelet/pods/5da9eada-18c2-4c38-a28a-e52262a5c20a/volumes"
Jul 28 22:59:31.241481 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:31.241442    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:31.342465 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:31.342431    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:31.505678 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:31.505597    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"]
Jul 28 22:59:31.505837 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:31.505810    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerName="istio-proxy" containerID="cri-o://e5ef94495e183bef4d90be8da387660c87d4aa34fae563b6b69e0be5ca12549b" gracePeriod=30
Jul 28 22:59:31.539730 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:31.539698    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:31.784189 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:31.784102    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:35.995510 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:35.995476    2583 patch_prober.go:28] interesting pod/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 22:59:35.995905 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:35.995540    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 22:59:36.765346 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.765286    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:59:36.921171 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.921134    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9k2w\" (UniqueName: \"kubernetes.io/projected/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-kube-api-access-l9k2w\") pod \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") "
Jul 28 22:59:36.921171 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.921181    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-envoy\") pod \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") "
Jul 28 22:59:36.921431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.921221    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-podinfo\") pod \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") "
Jul 28 22:59:36.921431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.921241    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istiod-ca-cert\") pod \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") "
Jul 28 22:59:36.921431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.921260    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-data\") pod \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") "
Jul 28 22:59:36.921431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.921286    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-workload-socket\") pod \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") "
Jul 28 22:59:36.921431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.921300    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-token\") pod \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") "
Jul 28 22:59:36.921431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.921353    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-workload-certs\") pod \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") "
Jul 28 22:59:36.921431 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.921372    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-credential-socket\") pod \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\" (UID: \"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad\") "
Jul 28 22:59:36.921891 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.921596    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" (UID: "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:36.921891 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.921747    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" (UID: "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:36.921891 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.921838    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" (UID: "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:36.922165 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.922091    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" (UID: "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:59:36.922240 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.922214    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-data" (OuterVolumeSpecName: "istio-data") pod "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" (UID: "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:36.923451 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.923423    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-kube-api-access-l9k2w" (OuterVolumeSpecName: "kube-api-access-l9k2w") pod "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" (UID: "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad"). InnerVolumeSpecName "kube-api-access-l9k2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:59:36.923451 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.923432    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-token" (OuterVolumeSpecName: "istio-token") pod "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" (UID: "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:59:36.923811 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.923780    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" (UID: "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 22:59:36.923811 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:36.923784    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" (UID: "cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:37.022986 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.022896    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-workload-socket\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:37.022986 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.022931    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-token\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:37.022986 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.022940    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-workload-certs\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:37.022986 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.022950    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-credential-socket\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:37.022986 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.022959    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9k2w\" (UniqueName: \"kubernetes.io/projected/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-kube-api-access-l9k2w\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:37.022986 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.022968    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-envoy\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:37.022986 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.022977    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-podinfo\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:37.022986 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.022985    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istiod-ca-cert\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:37.022986 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.022992    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad-istio-data\") on node \"ip-10-0-128-9.ec2.internal\" DevicePath \"\""
Jul 28 22:59:37.173499 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.173470    2583 generic.go:356] "Generic (PLEG): container finished" podID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerID="e5ef94495e183bef4d90be8da387660c87d4aa34fae563b6b69e0be5ca12549b" exitCode=0
Jul 28 22:59:37.173665 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.173524    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"
Jul 28 22:59:37.173665 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.173553    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" event={"ID":"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad","Type":"ContainerDied","Data":"e5ef94495e183bef4d90be8da387660c87d4aa34fae563b6b69e0be5ca12549b"}
Jul 28 22:59:37.173665 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.173609    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s" event={"ID":"cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad","Type":"ContainerDied","Data":"7d3a6f5c3ad45ce65fe326c9dddd557c134a332f8c5a5dc596ca20060a3b6595"}
Jul 28 22:59:37.173665 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.173630    2583 scope.go:117] "RemoveContainer" containerID="e5ef94495e183bef4d90be8da387660c87d4aa34fae563b6b69e0be5ca12549b"
Jul 28 22:59:37.185730 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.185712    2583 scope.go:117] "RemoveContainer" containerID="149ed76841a77673a189be4fa727f76a2c1510d6a17595a587d490cb7fcc89d4"
Jul 28 22:59:37.194065 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.194049    2583 scope.go:117] "RemoveContainer" containerID="e5ef94495e183bef4d90be8da387660c87d4aa34fae563b6b69e0be5ca12549b"
Jul 28 22:59:37.194305 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:59:37.194286    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5ef94495e183bef4d90be8da387660c87d4aa34fae563b6b69e0be5ca12549b\": container with ID starting with e5ef94495e183bef4d90be8da387660c87d4aa34fae563b6b69e0be5ca12549b not found: ID does not exist" containerID="e5ef94495e183bef4d90be8da387660c87d4aa34fae563b6b69e0be5ca12549b"
Jul 28 22:59:37.194363 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.194314    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5ef94495e183bef4d90be8da387660c87d4aa34fae563b6b69e0be5ca12549b"} err="failed to get container status \"e5ef94495e183bef4d90be8da387660c87d4aa34fae563b6b69e0be5ca12549b\": rpc error: code = NotFound desc = could not find container \"e5ef94495e183bef4d90be8da387660c87d4aa34fae563b6b69e0be5ca12549b\": container with ID starting with e5ef94495e183bef4d90be8da387660c87d4aa34fae563b6b69e0be5ca12549b not found: ID does not exist"
Jul 28 22:59:37.194363 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.194333    2583 scope.go:117] "RemoveContainer" containerID="149ed76841a77673a189be4fa727f76a2c1510d6a17595a587d490cb7fcc89d4"
Jul 28 22:59:37.194556 ip-10-0-128-9 kubenswrapper[2583]: E0728 22:59:37.194540    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"149ed76841a77673a189be4fa727f76a2c1510d6a17595a587d490cb7fcc89d4\": container with ID starting with 149ed76841a77673a189be4fa727f76a2c1510d6a17595a587d490cb7fcc89d4 not found: ID does not exist" containerID="149ed76841a77673a189be4fa727f76a2c1510d6a17595a587d490cb7fcc89d4"
Jul 28 22:59:37.194594 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.194563    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"149ed76841a77673a189be4fa727f76a2c1510d6a17595a587d490cb7fcc89d4"} err="failed to get container status \"149ed76841a77673a189be4fa727f76a2c1510d6a17595a587d490cb7fcc89d4\": rpc error: code = NotFound desc = could not find container \"149ed76841a77673a189be4fa727f76a2c1510d6a17595a587d490cb7fcc89d4\": container with ID starting with 149ed76841a77673a189be4fa727f76a2c1510d6a17595a587d490cb7fcc89d4 not found: ID does not exist"
Jul 28 22:59:37.197981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.197960    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"]
Jul 28 22:59:37.201861 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:37.201842    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-1661d9-openshift-default-5bdbbdff6-t455s"]
Jul 28 22:59:38.363379 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:38.363301    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d45996775-ngz2z_cf38cbba-f1b2-4ca0-82ef-c5989ee07b27/copy-manifests/0.log"
Jul 28 22:59:38.370413 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:38.370388    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d45996775-ngz2z_cf38cbba-f1b2-4ca0-82ef-c5989ee07b27/manager/0.log"
Jul 28 22:59:38.431043 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:38.431012    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" path="/var/lib/kubelet/pods/cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad/volumes"
Jul 28 22:59:38.978699 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:38.978670    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754615-88wvl_d4277813-1e72-4863-8eda-6e5fd847b40d/cleanup/0.log"
Jul 28 22:59:39.094085 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:39.094061    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754630-x96gm_8e0b0203-f96e-4223-b68c-ae5efa7eb9c2/cleanup/0.log"
Jul 28 22:59:39.211409 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:39.211380    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754645-pk5zr_7460df6e-7d95-4c3d-999c-3191081cf666/cleanup/0.log"
Jul 28 22:59:39.317050 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:39.316979    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-994mz_0c4b12f5-75a7-4c7d-987f-499ae220bbe7/postgres/0.log"
Jul 28 22:59:40.099237 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:40.099208    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn_41654737-66b5-44ea-8bbb-78c923e5af0e/extract/0.log"
Jul 28 22:59:40.105590 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:40.105563    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn_41654737-66b5-44ea-8bbb-78c923e5af0e/util/0.log"
Jul 28 22:59:40.111940 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:40.111915    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn_41654737-66b5-44ea-8bbb-78c923e5af0e/pull/0.log"
Jul 28 22:59:40.226496 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:40.226442    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p_5e996bb9-7b01-43e8-be2e-5f08f929e82c/util/0.log"
Jul 28 22:59:40.232496 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:40.232460    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p_5e996bb9-7b01-43e8-be2e-5f08f929e82c/pull/0.log"
Jul 28 22:59:40.238054 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:40.238033    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p_5e996bb9-7b01-43e8-be2e-5f08f929e82c/extract/0.log"
Jul 28 22:59:40.346101 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:40.346064    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9_4f987892-0ab9-416d-b4e3-1ccee76ef06b/util/0.log"
Jul 28 22:59:40.354046 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:40.353984    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9_4f987892-0ab9-416d-b4e3-1ccee76ef06b/pull/0.log"
Jul 28 22:59:40.359397 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:40.359380    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9_4f987892-0ab9-416d-b4e3-1ccee76ef06b/extract/0.log"
Jul 28 22:59:40.468725 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:40.468696    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v_6945deb3-3cc1-48e1-911a-2ec3e9c5fba2/util/0.log"
Jul 28 22:59:40.474967 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:40.474941    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v_6945deb3-3cc1-48e1-911a-2ec3e9c5fba2/pull/0.log"
Jul 28 22:59:40.480545 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:40.480524    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v_6945deb3-3cc1-48e1-911a-2ec3e9c5fba2/extract/0.log"
Jul 28 22:59:40.961661 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:40.961636    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-brlp8_37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad/kuadrant-console-plugin/0.log"
Jul 28 22:59:42.148010 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:42.147982    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cd5d84f76-xlct4_41bfaa5e-c4c1-4d52-a308-f759c8afdbcf/kube-auth-proxy/0.log"
Jul 28 22:59:42.614981 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:42.614952    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6597fb4cff-sdft5_5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e/router/0.log"
Jul 28 22:59:42.970705 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:42.970617    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-h7jxs_a1679b1b-8fff-4f18-9f3e-42438838654c/main/0.log"
Jul 28 22:59:43.304571 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:43.304479    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzbjs_77bf9758-8572-4c2d-9293-3f525ce7778e/main/0.log"
Jul 28 22:59:49.517557 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.517513    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lskt7/must-gather-g4vzb"]
Jul 28 22:59:49.518109 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.518047    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerName="istio-proxy"
Jul 28 22:59:49.518109 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.518067    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerName="istio-proxy"
Jul 28 22:59:49.518109 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.518094    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5da9eada-18c2-4c38-a28a-e52262a5c20a" containerName="revoke-keys"
Jul 28 22:59:49.518109 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.518102    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5da9eada-18c2-4c38-a28a-e52262a5c20a" containerName="revoke-keys"
Jul 28 22:59:49.518412 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.518235    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerName="istio-proxy"
Jul 28 22:59:49.518412 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.518254    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5da9eada-18c2-4c38-a28a-e52262a5c20a" containerName="revoke-keys"
Jul 28 22:59:49.518412 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.518346    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerName="istio-proxy"
Jul 28 22:59:49.518412 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.518360    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerName="istio-proxy"
Jul 28 22:59:49.518646 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.518465    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf7f8513-6bc2-4e6d-8ef0-e323b64b1fad" containerName="istio-proxy"
Jul 28 22:59:49.523109 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.523089    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lskt7/must-gather-g4vzb"
Jul 28 22:59:49.525068 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.525045    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lskt7\"/\"openshift-service-ca.crt\""
Jul 28 22:59:49.525613 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.525592    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lskt7\"/\"default-dockercfg-6vcgc\""
Jul 28 22:59:49.525698 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.525627    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lskt7\"/\"kube-root-ca.crt\""
Jul 28 22:59:49.531155 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.531131    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lskt7/must-gather-g4vzb"]
Jul 28 22:59:49.535326 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.535298    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:49.597264 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.597227    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log" limit=1024
Jul 28 22:59:49.639463 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.639430    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0d9478dd-3e5e-4b00-bbe0-82ee53413e11-must-gather-output\") pod \"must-gather-g4vzb\" (UID: \"0d9478dd-3e5e-4b00-bbe0-82ee53413e11\") " pod="openshift-must-gather-lskt7/must-gather-g4vzb"
Jul 28 22:59:49.639628 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.639476    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6wht\" (UniqueName: \"kubernetes.io/projected/0d9478dd-3e5e-4b00-bbe0-82ee53413e11-kube-api-access-p6wht\") pod \"must-gather-g4vzb\" (UID: \"0d9478dd-3e5e-4b00-bbe0-82ee53413e11\") " pod="openshift-must-gather-lskt7/must-gather-g4vzb"
Jul 28 22:59:49.740325 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.740289    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0d9478dd-3e5e-4b00-bbe0-82ee53413e11-must-gather-output\") pod \"must-gather-g4vzb\" (UID: \"0d9478dd-3e5e-4b00-bbe0-82ee53413e11\") " pod="openshift-must-gather-lskt7/must-gather-g4vzb"
Jul 28 22:59:49.740500 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.740340    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6wht\" (UniqueName: \"kubernetes.io/projected/0d9478dd-3e5e-4b00-bbe0-82ee53413e11-kube-api-access-p6wht\") pod \"must-gather-g4vzb\" (UID: \"0d9478dd-3e5e-4b00-bbe0-82ee53413e11\") " pod="openshift-must-gather-lskt7/must-gather-g4vzb"
Jul 28 22:59:49.740657 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.740636    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0d9478dd-3e5e-4b00-bbe0-82ee53413e11-must-gather-output\") pod \"must-gather-g4vzb\" (UID: \"0d9478dd-3e5e-4b00-bbe0-82ee53413e11\") " pod="openshift-must-gather-lskt7/must-gather-g4vzb"
Jul 28 22:59:49.748944 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.748924    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6wht\" (UniqueName: \"kubernetes.io/projected/0d9478dd-3e5e-4b00-bbe0-82ee53413e11-kube-api-access-p6wht\") pod \"must-gather-g4vzb\" (UID: \"0d9478dd-3e5e-4b00-bbe0-82ee53413e11\") " pod="openshift-must-gather-lskt7/must-gather-g4vzb"
Jul 28 22:59:49.833223 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.833133    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lskt7/must-gather-g4vzb"
Jul 28 22:59:49.981508 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.981476    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lskt7/must-gather-g4vzb"]
Jul 28 22:59:49.984699 ip-10-0-128-9 kubenswrapper[2583]: W0728 22:59:49.984672    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d9478dd_3e5e_4b00_bbe0_82ee53413e11.slice/crio-c4c1d0c42d89317a2fcbe63e8157a47bd40e811201d885ba9b134e70ebd31b00 WatchSource:0}: Error finding container c4c1d0c42d89317a2fcbe63e8157a47bd40e811201d885ba9b134e70ebd31b00: Status 404 returned error can't find the container with id c4c1d0c42d89317a2fcbe63e8157a47bd40e811201d885ba9b134e70ebd31b00
Jul 28 22:59:49.986510 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:49.986495    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:59:50.250014 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:50.249976    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lskt7/must-gather-g4vzb" event={"ID":"0d9478dd-3e5e-4b00-bbe0-82ee53413e11","Type":"ContainerStarted","Data":"c4c1d0c42d89317a2fcbe63e8157a47bd40e811201d885ba9b134e70ebd31b00"}
Jul 28 22:59:51.258205 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:51.258173    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lskt7/must-gather-g4vzb" event={"ID":"0d9478dd-3e5e-4b00-bbe0-82ee53413e11","Type":"ContainerStarted","Data":"74bf96548b356cf973fa6f2cdb5f99487f0f08dc27961180cdfc3c97d2f574e6"}
Jul 28 22:59:51.258205 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:51.258212    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lskt7/must-gather-g4vzb" event={"ID":"0d9478dd-3e5e-4b00-bbe0-82ee53413e11","Type":"ContainerStarted","Data":"507b3b0d83116ba978b88691f8b8e4321a286c1558c402720de5e61ad3b1ff41"}
Jul 28 22:59:51.281185 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:51.281115    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lskt7/must-gather-g4vzb" podStartSLOduration=1.509752878 podStartE2EDuration="2.281092401s" totalImagesPullingTime="771.339523ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:59:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:59:51.273333851 +0000 UTC m=+3335.281199410" watchObservedRunningTime="2026-07-28 22:59:51.281092401 +0000 UTC m=+3335.288957962"
Jul 28 22:59:52.416102 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:52.416070    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t5bf9_dae75d3b-5868-4cd2-b7e4-45f6a145b875/global-pull-secret-syncer/0.log"
Jul 28 22:59:52.473244 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:52.473201    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4hv59_939e5bee-11fa-4cc1-852b-49fba60700ee/konnectivity-agent/0.log"
Jul 28 22:59:52.580038 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:52.580004    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-9.ec2.internal_41dbbb64873b28a94876546d4a4c4566/haproxy/0.log"
Jul 28 22:59:56.765591 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:56.765557    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn_41654737-66b5-44ea-8bbb-78c923e5af0e/extract/0.log"
Jul 28 22:59:56.787744 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:56.787709    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn_41654737-66b5-44ea-8bbb-78c923e5af0e/util/0.log"
Jul 28 22:59:56.812451 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:56.812423    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gkzn_41654737-66b5-44ea-8bbb-78c923e5af0e/pull/0.log"
Jul 28 22:59:56.846507 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:56.846421    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p_5e996bb9-7b01-43e8-be2e-5f08f929e82c/extract/0.log"
Jul 28 22:59:56.868841 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:56.868788    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p_5e996bb9-7b01-43e8-be2e-5f08f929e82c/util/0.log"
Jul 28 22:59:56.894305 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:56.894270    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q4w9p_5e996bb9-7b01-43e8-be2e-5f08f929e82c/pull/0.log"
Jul 28 22:59:56.923216 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:56.923032    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9_4f987892-0ab9-416d-b4e3-1ccee76ef06b/extract/0.log"
Jul 28 22:59:56.944498 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:56.944473    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9_4f987892-0ab9-416d-b4e3-1ccee76ef06b/util/0.log"
Jul 28 22:59:56.964858 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:56.964783    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jbrk9_4f987892-0ab9-416d-b4e3-1ccee76ef06b/pull/0.log"
Jul 28 22:59:56.990148 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:56.990103    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v_6945deb3-3cc1-48e1-911a-2ec3e9c5fba2/extract/0.log"
Jul 28 22:59:57.011525 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:57.011432    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v_6945deb3-3cc1-48e1-911a-2ec3e9c5fba2/util/0.log"
Jul 28 22:59:57.033299 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:57.033180    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nws9v_6945deb3-3cc1-48e1-911a-2ec3e9c5fba2/pull/0.log"
Jul 28 22:59:57.798598 ip-10-0-128-9 kubenswrapper[2583]: I0728 22:59:57.798507    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-brlp8_37eb6299-0b0d-48c3-bb38-e0c6cc5b1bad/kuadrant-console-plugin/0.log"
Jul 28 23:00:00.357465 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:00.357426    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-97gkk_7bc1592a-3b91-40a0-979c-2799511bcbda/cluster-monitoring-operator/0.log"
Jul 28 23:00:00.632371 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:00.632343    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mrpm6_6401cb70-70af-4012-acfc-a753e352f84d/node-exporter/0.log"
Jul 28 23:00:00.651698 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:00.651661    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mrpm6_6401cb70-70af-4012-acfc-a753e352f84d/kube-rbac-proxy/0.log"
Jul 28 23:00:00.673898 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:00.673862    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mrpm6_6401cb70-70af-4012-acfc-a753e352f84d/init-textfile/0.log"
Jul 28 23:00:01.092538 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:01.092451    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-59b996856f-7qjkh_7aa44009-6562-41c7-924d-8a530fd4d38d/telemeter-client/0.log"
Jul 28 23:00:01.113609 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:01.113577    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-59b996856f-7qjkh_7aa44009-6562-41c7-924d-8a530fd4d38d/reload/0.log"
Jul 28 23:00:01.148352 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:01.148316    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-59b996856f-7qjkh_7aa44009-6562-41c7-924d-8a530fd4d38d/kube-rbac-proxy/0.log"
Jul 28 23:00:02.487609 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:02.487582    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-sswtl_4f11e7df-616c-49a2-9071-0f32a5b25adf/networking-console-plugin/0.log"
Jul 28 23:00:03.044195 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.044154    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"]
Jul 28 23:00:03.051106 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.051075    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.054548 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.054422    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"]
Jul 28 23:00:03.086403 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.086365    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1bd626a7-f417-435c-b495-fc6f436bbed5-proc\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.086721 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.086702    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1bd626a7-f417-435c-b495-fc6f436bbed5-sys\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.086856 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.086839    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1bd626a7-f417-435c-b495-fc6f436bbed5-lib-modules\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.086973 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.086960    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1bd626a7-f417-435c-b495-fc6f436bbed5-podres\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.087125 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.087111    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcsrw\" (UniqueName: \"kubernetes.io/projected/1bd626a7-f417-435c-b495-fc6f436bbed5-kube-api-access-vcsrw\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.169935 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.169881    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/1.log"
Jul 28 23:00:03.175296 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.175168    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-57xd4_095ffcbc-5ca5-483e-b0d5-c919b4b9ab88/console-operator/2.log"
Jul 28 23:00:03.194302 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.189305    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1bd626a7-f417-435c-b495-fc6f436bbed5-sys\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.194302 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.189415    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1bd626a7-f417-435c-b495-fc6f436bbed5-sys\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.194302 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.189464    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1bd626a7-f417-435c-b495-fc6f436bbed5-lib-modules\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.194302 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.189594    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1bd626a7-f417-435c-b495-fc6f436bbed5-lib-modules\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.194302 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.189645    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1bd626a7-f417-435c-b495-fc6f436bbed5-podres\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.194302 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.189705    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcsrw\" (UniqueName: \"kubernetes.io/projected/1bd626a7-f417-435c-b495-fc6f436bbed5-kube-api-access-vcsrw\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.194302 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.189745    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1bd626a7-f417-435c-b495-fc6f436bbed5-proc\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.194302 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.189865    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1bd626a7-f417-435c-b495-fc6f436bbed5-proc\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.194302 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.189949    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1bd626a7-f417-435c-b495-fc6f436bbed5-podres\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.200468 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.199242    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcsrw\" (UniqueName: \"kubernetes.io/projected/1bd626a7-f417-435c-b495-fc6f436bbed5-kube-api-access-vcsrw\") pod \"perf-node-gather-daemonset-9lz5f\" (UID: \"1bd626a7-f417-435c-b495-fc6f436bbed5\") " pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.370755 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.370725    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:03.552114 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.552086    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"]
Jul 28 23:00:03.730125 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.730095    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-86565574f-rrnkj_5f47692c-c93b-4c33-ba0d-f58fc617f9ce/console/0.log"
Jul 28 23:00:03.774125 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:03.774090    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-llh92_08aa7923-c078-4a4f-9577-55ccac85fe3a/download-server/0.log"
Jul 28 23:00:04.350429 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:04.350402    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-th52b_0fcee02e-7d41-4794-96bf-9d49280416c3/volume-data-source-validator/0.log"
Jul 28 23:00:04.371146 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:04.371019    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f" event={"ID":"1bd626a7-f417-435c-b495-fc6f436bbed5","Type":"ContainerStarted","Data":"a96b4e88b0ed0f0bb78d9aa6aad105de407888899acb15c0b7beaea445920aa8"}
Jul 28 23:00:04.371348 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:04.371240    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:04.371348 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:04.371287    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f" event={"ID":"1bd626a7-f417-435c-b495-fc6f436bbed5","Type":"ContainerStarted","Data":"e8b1486843f49b27e32fba93e4145022c4fcbeac9a0aa4a1ed3f667fab411bf2"}
Jul 28 23:00:04.390345 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:04.390285    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f" podStartSLOduration=1.39026284 podStartE2EDuration="1.39026284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:00:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:00:04.389284126 +0000 UTC m=+3348.397149702" watchObservedRunningTime="2026-07-28 23:00:04.39026284 +0000 UTC m=+3348.398128390"
Jul 28 23:00:05.412217 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:05.412184    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cqbpk_8b67f675-10f2-4616-85c9-7feec90a410c/dns/0.log"
Jul 28 23:00:05.433635 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:05.432867    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cqbpk_8b67f675-10f2-4616-85c9-7feec90a410c/kube-rbac-proxy/0.log"
Jul 28 23:00:05.586150 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:05.586124    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lkmdh_769507e6-3089-4849-865d-283919d41c01/dns-node-resolver/0.log"
Jul 28 23:00:06.222657 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:06.222627    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zvgg7_63b7aaeb-cc08-452a-89b7-59cf107dce99/node-ca/0.log"
Jul 28 23:00:06.305230 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:06.305187    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl"]
Jul 28 23:00:06.308817 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:06.308773    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754615-88wvl"]
Jul 28 23:00:06.432038 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:06.432003    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4277813-1e72-4863-8eda-6e5fd847b40d" path="/var/lib/kubelet/pods/d4277813-1e72-4863-8eda-6e5fd847b40d/volumes"
Jul 28 23:00:07.369933 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:07.369904    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cd5d84f76-xlct4_41bfaa5e-c4c1-4d52-a308-f759c8afdbcf/kube-auth-proxy/0.log"
Jul 28 23:00:07.545096 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:07.545068    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6597fb4cff-sdft5_5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e/router/0.log"
Jul 28 23:00:08.352729 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:08.352698    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hjds4_ac77877d-96a7-41ed-b29d-6e3dccfdecf3/serve-healthcheck-canary/0.log"
Jul 28 23:00:08.865818 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:08.865769    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-77hcw_1465e125-b955-4c39-bc95-67661bd1f3a3/insights-operator/0.log"
Jul 28 23:00:08.868192 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:08.868144    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-77hcw_1465e125-b955-4c39-bc95-67661bd1f3a3/insights-operator/1.log"
Jul 28 23:00:09.032854 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:09.032824    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-npvph_76b97389-7eb1-46ac-8aa9-cb86280b26db/kube-rbac-proxy/0.log"
Jul 28 23:00:09.052488 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:09.052453    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-npvph_76b97389-7eb1-46ac-8aa9-cb86280b26db/exporter/0.log"
Jul 28 23:00:09.074154 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:09.074129    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-npvph_76b97389-7eb1-46ac-8aa9-cb86280b26db/extractor/0.log"
Jul 28 23:00:10.390690 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:10.390657    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lskt7/perf-node-gather-daemonset-9lz5f"
Jul 28 23:00:12.066715 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:12.066679    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d45996775-ngz2z_cf38cbba-f1b2-4ca0-82ef-c5989ee07b27/manager/0.log"
Jul 28 23:00:12.085771 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:12.085743    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d45996775-ngz2z_cf38cbba-f1b2-4ca0-82ef-c5989ee07b27/copy-manifests/0.log"
Jul 28 23:00:12.543976 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:12.543938    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cd5d84f76-xlct4_41bfaa5e-c4c1-4d52-a308-f759c8afdbcf/kube-auth-proxy/0.log"
Jul 28 23:00:12.678456 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:12.678418    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6597fb4cff-sdft5_5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e/router/0.log"
Jul 28 23:00:13.613211 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:13.613166    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-64ddd775c9-chjj4_25e6fd4a-cec6-4160-8556-69f4ba320818/manager/0.log"
Jul 28 23:00:13.637572 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:13.637527    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-d7ll9_36c779c4-7c38-4290-a263-996940b9b881/openshift-lws-operator/0.log"
Jul 28 23:00:14.346553 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:14.346518    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cd5d84f76-xlct4_41bfaa5e-c4c1-4d52-a308-f759c8afdbcf/kube-auth-proxy/0.log"
Jul 28 23:00:14.491339 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:14.491233    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6597fb4cff-sdft5_5b4bd6f3-04c2-4a36-b0f7-f2ed0fc9914e/router/0.log"
Jul 28 23:00:17.013016 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:17.012986    2583 scope.go:117] "RemoveContainer" containerID="ba8adadc19f171b4184611618373ed621f4b667aedb5b4753d1018f49d58162b"
Jul 28 23:00:18.484024 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:18.483993    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-lnrxf_1fabc7ab-7d49-45c8-8757-00dbab295cdb/migrator/0.log"
Jul 28 23:00:18.504325 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:18.504296    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-lnrxf_1fabc7ab-7d49-45c8-8757-00dbab295cdb/graceful-termination/0.log"
Jul 28 23:00:18.880252 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:18.880215    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p_3e9b3b23-0e89-4d3f-8f19-6830b0b68215/kube-storage-version-migrator-operator/1.log"
Jul 28 23:00:18.881890 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:18.881856    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-qqc8p_3e9b3b23-0e89-4d3f-8f19-6830b0b68215/kube-storage-version-migrator-operator/0.log"
Jul 28 23:00:20.231407 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:20.231338    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjwtm_2a85189c-c2ee-4a08-bb3e-7ad54914bc63/kube-multus-additional-cni-plugins/0.log"
Jul 28 23:00:20.251472 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:20.251428    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjwtm_2a85189c-c2ee-4a08-bb3e-7ad54914bc63/egress-router-binary-copy/0.log"
Jul 28 23:00:20.271316 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:20.271240    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjwtm_2a85189c-c2ee-4a08-bb3e-7ad54914bc63/cni-plugins/0.log"
Jul 28 23:00:20.294345 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:20.294311    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjwtm_2a85189c-c2ee-4a08-bb3e-7ad54914bc63/bond-cni-plugin/0.log"
Jul 28 23:00:20.315935 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:20.315906    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjwtm_2a85189c-c2ee-4a08-bb3e-7ad54914bc63/routeoverride-cni/0.log"
Jul 28 23:00:20.336585 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:20.336556    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjwtm_2a85189c-c2ee-4a08-bb3e-7ad54914bc63/whereabouts-cni-bincopy/0.log"
Jul 28 23:00:20.355284 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:20.355164    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjwtm_2a85189c-c2ee-4a08-bb3e-7ad54914bc63/whereabouts-cni/0.log"
Jul 28 23:00:20.443435 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:20.443400    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wqv74_928ed8af-4599-402e-8dc6-5a3ca176d89f/kube-multus/0.log"
Jul 28 23:00:20.468705 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:20.468675    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5vh87_33b4bdc7-694b-45e4-8c88-3cd9f134865d/network-metrics-daemon/0.log"
Jul 28 23:00:20.485731 ip-10-0-128-9 kubenswrapper[2583]: I0728 23:00:20.485637    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5vh87_33b4bdc7-694b-45e4-8c88-3cd9f134865d/kube-rbac-proxy/0.log"