Jul 15 04:58:37.305661 ip-10-0-128-231 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 04:58:37.594098 ip-10-0-128-231 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 04:58:37.594098 ip-10-0-128-231 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 04:58:37.594098 ip-10-0-128-231 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 04:58:37.594098 ip-10-0-128-231 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 04:58:37.594098 ip-10-0-128-231 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 04:58:37.594682 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.594531    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598913    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598928    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598935    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598940    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598945    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598950    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598955    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598960    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598964    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598968    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598972    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598976    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598983    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598987    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598991    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598995    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.598999    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599003    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599007    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:58:37.838190 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599026    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:58:37.655988 ip-10-0-128-231 systemd[1]: Started Kubernetes Kubelet.
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599030    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599034    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599039    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599043    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599047    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599051    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599055    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599060    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599064    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599068    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599071    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599075    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599079    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599082    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599086    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599089    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599093    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599097    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599101    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:58:37.851393 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599105    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599109    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599113    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599116    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599121    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599125    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599129    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599133    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599137    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599140    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599144    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599148    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599151    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599155    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599159    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599163    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599168    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599173    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599177    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599182    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:58:37.852460 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599186    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599190    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599196    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599202    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599208    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599213    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599218    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599222    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599226    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599230    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599234    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599238    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599245    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599249    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599253    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599257    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599261    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599265    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599270    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:58:37.853778 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599274    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599278    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599281    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599285    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599290    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599295    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599300    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599304    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599309    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599313    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599318    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599322    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599329    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599332    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599337    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599340    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599344    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599348    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599352    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:58:37.854797 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599356    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.599360    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600884    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600894    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600900    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600904    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600908    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600912    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600916    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600920    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600924    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600928    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600931    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600936    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600939    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600943    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600948    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600952    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600956    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600961    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:58:37.855873 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600964    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600968    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600971    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600975    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600978    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600982    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600986    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600989    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600993    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.600997    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601000    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601004    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601008    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601011    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601015    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601019    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601023    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601027    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601030    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601033    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:58:37.856934 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601049    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601053    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601058    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601061    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601065    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601069    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601075    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601079    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601084    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601089    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601093    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601097    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601101    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601104    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601108    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601114    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601118    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601122    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601125    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:58:37.857998 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601129    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601133    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601137    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601140    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601144    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601147    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601152    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601155    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601159    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601164    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601169    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601173    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601177    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601181    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601191    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601195    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601199    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601203    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601207    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601210    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:58:37.858826 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601214    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601217    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601222    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601225    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601229    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601232    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601236    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601239    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601243    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601248    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601255    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601259    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601263    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601266    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601270    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601274    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601277    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601281    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601284    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601288    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:58:37.859909 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601291    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601295    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601825    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601844    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601848    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601853    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601859    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601864    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601868    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601874    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601878    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601882    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601885    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601889    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601893    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601897    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601900    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601904    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601908    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:58:37.860950 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601912    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601916    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601919    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601923    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601926    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601930    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601934    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601938    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601941    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601945    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601948    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601952    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601956    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601959    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601963    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601967    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601972    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601975    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601979    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601983    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:58:37.862011 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601987    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601992    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601995    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.601999    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602003    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602006    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602009    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602013    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602017    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602021    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602024    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602028    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602032    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602035    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602039    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602043    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602047    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602050    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602053    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:58:37.862887 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602057    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602061    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602064    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602068    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602072    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602076    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602080    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602084    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602089    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602093    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602096    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602100    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602104    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602107    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602112    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602116    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602119    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602122    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602127    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:58:37.863879 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602130    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602136    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602148    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602152    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602156    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602161    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602165    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602169    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602172    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602175    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602179    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602183    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602187    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602191    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602195    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602199    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602202    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602206    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602210    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602213    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:58:37.864728 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602217    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602220    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602224    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.602227    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602322    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602331    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602340    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602346    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602359    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602364    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602388    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602395    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602401    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602406    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602411    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602415    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602420    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602424    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602428    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602433    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602437    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602441    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602449    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602454    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 04:58:37.865578 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602458    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602463    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602468    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602474    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602478    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602483    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602487    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602492    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602496    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602500    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602504    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602509    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602520    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602524    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602528    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602533    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602539    2562 flags.go:64] FLAG: --enable-server="true"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602544    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602550    2562 flags.go:64] FLAG: --event-burst="100"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602555    2562 flags.go:64] FLAG: --event-qps="50"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602560    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602564    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602568    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602574    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 04:58:37.866652 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602578    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602582    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602587    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602591    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602595    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602602    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602607    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602612    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602616    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602620    2562 flags.go:64] FLAG: --feature-gates=""
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602626    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602631    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602635    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602640    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602645    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602649    2562 flags.go:64] FLAG: --help="false"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602653    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-128-231.ec2.internal"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602658    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602662    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602666    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602671    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602676    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602680    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 04:58:37.867709 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602685    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602690    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602694    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602700    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602704    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602708    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602712    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602716    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602721    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602725    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602729    2562 flags.go:64] FLAG: --lock-file=""
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602732    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602736    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602740    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602749    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602753    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602757    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602761    2562 flags.go:64] FLAG: --logging-format="text"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602765    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602770    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602774    2562 flags.go:64] FLAG: --manifest-url=""
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602778    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602785    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602789    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602796    2562 flags.go:64] FLAG: --max-pods="110"
Jul 15 04:58:37.868793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602800    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602804    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602808    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602812    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602816    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602821    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602825    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602835    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602841    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602845    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602850    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602855    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602863    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602867    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602871    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602875    2562 flags.go:64] FLAG: --port="10250"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602879    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602884    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ecf294da10dc45ed"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602888    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602892    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602897    2562 flags.go:64] FLAG: --register-node="true"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602903    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602908    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602912    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 15 04:58:37.869853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602917    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602921    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602926    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602931    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602936    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602940    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602944    2562 flags.go:64] FLAG: --runonce="false"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602948    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602953    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602957    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602961    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602965    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602969    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602974    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602978    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602982    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602987    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602992    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.602996    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603001    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603006    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603011    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603018    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603022    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603027    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 04:58:37.872069 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603033    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603037    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603041    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603046    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603051    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603055    2562 flags.go:64] FLAG: --v="2"
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603061    2562 flags.go:64] FLAG: --version="false"
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603067    2562 flags.go:64] FLAG: --vmodule=""
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603072    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.603077    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603233    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603238    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603242    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603246    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603251    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603255    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603259    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603263    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603266    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603270    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603274    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603278    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:58:37.872808 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603282    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603286    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603292    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603295    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603300    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603304    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603311    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603317    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603321    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603325    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603329    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603333    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603336    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603341    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603346    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603351    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603355    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603359    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603363    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603380    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:58:38.183385 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603385    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603395    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603399    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603403    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603407    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603411    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603414    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603418    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603421    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603425    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603428    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603432    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603436    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603439    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603445    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603449    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603453    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603457    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603460    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603464    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:58:38.184437 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603468    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603471    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603475    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603481    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603485    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603490    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603495    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603499    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603503    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603506    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603510    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603513    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603517    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603520    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603524    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603528    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603532    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603535    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:58:38.185094 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603538    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603542    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603546    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603551    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603557    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603561    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603572    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603576    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603582    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603585    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603589    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603592    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603596    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603600    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603605    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603609    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603613    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603616    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603620    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:58:38.185661 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603624    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603630    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603633    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603636    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603639    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603643    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603647    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603651    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603654    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.603658    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.604107    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.611182    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.611197    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.611216    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.611223    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 04:58:38.186203 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611277    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611283    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611288    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611292    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611295    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611297    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611300    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611303    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611306    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611308    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611311    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611316    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611319    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611322    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611326    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611329    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611331    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611334    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611337    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:58:38.186727 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611340    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611342    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611345    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611348    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611351    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611354    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611356    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611359    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611361    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611363    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611383    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611387    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611390    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611392    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611395    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611398    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611401    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611404    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611406    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:58:38.187292 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611409    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611411    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611414    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611417    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611422    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611425    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611428    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611431    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611433    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611436    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611439    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611441    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611444    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611446    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611449    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611456    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611458    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611461    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611463    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611466    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:58:38.187896 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611469    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611471    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611474    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611477    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611480    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611482    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611485    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611487    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611490    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611492    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611495    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611497    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611500    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611502    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611505    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611508    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611512    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611515    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611517    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611520    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:58:38.188471 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611522    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611525    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611527    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611530    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611533    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611535    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611538    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611542    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611545    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611547    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611550    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611552    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611555    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611558    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611560    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611563    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611565    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611568    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611570    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611573    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:58:38.189078 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611575    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.611581    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611696    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611701    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611704    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611706    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611709    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611712    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611715    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611719    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611722    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611724    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611727    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611730    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611732    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 04:58:38.189681 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611735    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611737    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611740    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611742    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611746    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611750    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611752    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611755    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611757    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611760    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611763    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611765    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611768    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611771    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611774    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611777    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611779    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611782    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611784    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 04:58:38.190226 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611786    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611789    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611791    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611794    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611796    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611799    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611801    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611805    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611808    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611810    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611813    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611815    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611818    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611821    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611823    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611826    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611828    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611832    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611834    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 04:58:38.190807 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611837    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611839    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611841    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611844    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611847    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611849    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611852    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611854    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611856    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611859    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611862    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611864    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611867    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611869    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611871    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611874    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611876    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611880    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611883    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 04:58:38.191381 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611886    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611890    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611894    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611896    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611899    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611902    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611904    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611906    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611909    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611912    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611915    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611918    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611921    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611925    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611928    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611930    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611933    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611935    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611938    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 04:58:38.191960 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611941    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611943    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611945    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611948    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611950    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611953    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611956    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611958    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611961    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:37.611963    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.611968    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.612078    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.613893    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.614825    2562 server.go:1019] "Starting client certificate rotation"
Jul 15 04:58:38.192548 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.614920    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 04:58:38.192960 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.614984    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 04:58:38.192960 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.616646    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 04:58:38.192960 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.619937    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 04:58:38.192960 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.628358    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 15 04:58:38.192960 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.630124    2562 log.go:25] "Validated CRI v1 image API"
Jul 15 04:58:38.192960 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.631343    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 04:58:38.192960 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.633238    2562 fs.go:135] Filesystem UUIDs: map[3311ca3a-8393-4edb-ad57-ef171c83469a:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a66c56ca-18da-4d4e-a3b2-fa8496956f90:/dev/nvme0n1p4]
Jul 15 04:58:38.192960 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.633257    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.638409    2562 manager.go:216] Machine: {Timestamp:2026-07-15 04:58:37.637285501 +0000 UTC m=+0.240838100 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3190278 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec208a53e7052c82f84ed2d7d455f31e SystemUUID:ec208a53-e705-2c82-f84e-d2d7d455f31e BootID:e7c8c355-ab24-4156-ba22-5e5476429ee8 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6a:4d:a2:1e:cb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6a:4d:a2:1e:cb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:1a:4d:4f:fa:b7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.638515    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.638601    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.640529    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.640554    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-231.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.641051    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.641059    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.641067    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.641492    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.642386    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.645630    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.645732    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.645765    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.645788    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.645798    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.645814    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 04:58:38.193325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.645824    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.645833    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.646176    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.646191    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.647693    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649145    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649172    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649312    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649326    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649332    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649337    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649345    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649350    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649633    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649639    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649647    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649653    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649662    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.649669    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.650135    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.650149    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.653417    2562 server.go:1268] "Started kubelet"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.653540    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.653571    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.655640    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.656633    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.658138    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.660993    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-231.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.661447    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.661497    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-231.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.661883    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.664671    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.664737    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.664747    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.665145    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.665189    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.665205    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.665271    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.665282    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.665569    2562 factory.go:55] Registering systemd factory
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.665601    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.665871    2562 factory.go:153] Registering CRI-O factory
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.665882    2562 factory.go:223] Registration of the crio container factory successfully
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.665934    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.665959    2562 factory.go:103] Registering Raw factory
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.665977    2562 manager.go:1196] Started watching for new ooms in manager
Jul 15 04:58:38.193907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.666483    2562 manager.go:349] Starting recovery of all containers
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.672197    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.672304    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-231.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.672312    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-231.ec2.internal.18c25dc86484980b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-231.ec2.internal,UID:ip-10-0-128-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-231.ec2.internal,},FirstTimestamp:2026-07-15 04:58:37.653391371 +0000 UTC m=+0.256943969,LastTimestamp:2026-07-15 04:58:37.653391371 +0000 UTC m=+0.256943969,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-231.ec2.internal,}"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.675988    2562 manager.go:354] Recovery completed
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.684261    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.686039    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.686082    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.686094    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.686631    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.686638    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.686640    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.686704    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.687292    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.687316    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.687326    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.687962    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-231.ec2.internal.18c25dc866772474  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-231.ec2.internal,UID:ip-10-0-128-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-231.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-231.ec2.internal,},FirstTimestamp:2026-07-15 04:58:37.686064244 +0000 UTC m=+0.289616847,LastTimestamp:2026-07-15 04:58:37.686064244 +0000 UTC m=+0.289616847,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-231.ec2.internal,}"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.688941    2562 policy_none.go:49] "None policy: Start"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.688954    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.688963    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.691012    2562 policy_none.go:47] "Start"
Jul 15 04:58:38.489027 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.698718    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-231.ec2.internal.18c25dc866777fb9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-231.ec2.internal,UID:ip-10-0-128-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-231.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-231.ec2.internal,},FirstTimestamp:2026-07-15 04:58:37.686087609 +0000 UTC m=+0.289640207,LastTimestamp:2026-07-15 04:58:37.686087609 +0000 UTC m=+0.289640207,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-231.ec2.internal,}"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.710091    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-231.ec2.internal.18c25dc86677a9ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-231.ec2.internal,UID:ip-10-0-128-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-231.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-231.ec2.internal,},FirstTimestamp:2026-07-15 04:58:37.686098382 +0000 UTC m=+0.289650981,LastTimestamp:2026-07-15 04:58:37.686098382 +0000 UTC m=+0.289650981,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-231.ec2.internal,}"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.719131    2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-231.ec2.internal.18c25dc866772474\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-231.ec2.internal.18c25dc866772474  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-231.ec2.internal,UID:ip-10-0-128-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-231.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-231.ec2.internal,},FirstTimestamp:2026-07-15 04:58:37.686064244 +0000 UTC m=+0.289616847,LastTimestamp:2026-07-15 04:58:37.687302724 +0000 UTC m=+0.290855322,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-231.ec2.internal,}"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.727208    2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-231.ec2.internal.18c25dc866777fb9\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-231.ec2.internal.18c25dc866777fb9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-231.ec2.internal,UID:ip-10-0-128-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-231.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-231.ec2.internal,},FirstTimestamp:2026-07-15 04:58:37.686087609 +0000 UTC m=+0.289640207,LastTimestamp:2026-07-15 04:58:37.687320276 +0000 UTC m=+0.290872875,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-231.ec2.internal,}"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.731962    2562 manager.go:337] "Starting Device Plugin manager"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.732001    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.732011    2562 server.go:87] "Starting device plugin registration server"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.732300    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.732316    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.732412    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.732496    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.732504    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.733036    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.733094    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.736067    2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-231.ec2.internal.18c25dc86677a9ce\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-231.ec2.internal.18c25dc86677a9ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-231.ec2.internal,UID:ip-10-0-128-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-231.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-231.ec2.internal,},FirstTimestamp:2026-07-15 04:58:37.686098382 +0000 UTC m=+0.289650981,LastTimestamp:2026-07-15 04:58:37.687329632 +0000 UTC m=+0.290882231,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-231.ec2.internal,}"
Jul 15 04:58:38.490490 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.742869    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-blpp2"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.744738    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-231.ec2.internal.18c25dc8695a08e4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-231.ec2.internal,UID:ip-10-0-128-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-128-231.ec2.internal,},FirstTimestamp:2026-07-15 04:58:37.734488292 +0000 UTC m=+0.338040883,LastTimestamp:2026-07-15 04:58:37.734488292 +0000 UTC m=+0.338040883,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-231.ec2.internal,}"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.750530    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-blpp2"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.833031    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.834119    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.834156    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.834166    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.834953    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.837917    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.838790    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.838822    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.838862    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.838901    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.846498    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.852925    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.852944    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-231.ec2.internal\": node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.892536    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.939896    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-231.ec2.internal"]
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.939990    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.941555    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.941581    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.941592    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.943739    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.943845    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.943872    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.944494    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.944499    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.944519    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.944532    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.944520    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.944595    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.946719    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.946743    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.947428    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.947448    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:38.491821 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.947457    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.966579    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd9a57e0221e381594f2ffabc783a23f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal\" (UID: \"cd9a57e0221e381594f2ffabc783a23f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.966612    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33e99f318c4c76ca78a270f6e363c8e1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-231.ec2.internal\" (UID: \"33e99f318c4c76ca78a270f6e363c8e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:37.966628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cd9a57e0221e381594f2ffabc783a23f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal\" (UID: \"cd9a57e0221e381594f2ffabc783a23f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.977090    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-231.ec2.internal\" not found" node="ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.983657    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-231.ec2.internal\" not found" node="ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:37.993385    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.067127    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cd9a57e0221e381594f2ffabc783a23f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal\" (UID: \"cd9a57e0221e381594f2ffabc783a23f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.067162    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd9a57e0221e381594f2ffabc783a23f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal\" (UID: \"cd9a57e0221e381594f2ffabc783a23f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.067178    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33e99f318c4c76ca78a270f6e363c8e1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-231.ec2.internal\" (UID: \"33e99f318c4c76ca78a270f6e363c8e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.067223    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33e99f318c4c76ca78a270f6e363c8e1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-231.ec2.internal\" (UID: \"33e99f318c4c76ca78a270f6e363c8e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.067227    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cd9a57e0221e381594f2ffabc783a23f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal\" (UID: \"cd9a57e0221e381594f2ffabc783a23f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.067229    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd9a57e0221e381594f2ffabc783a23f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal\" (UID: \"cd9a57e0221e381594f2ffabc783a23f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:38.095331    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:38.197301    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.278514    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.285484    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:38.298436    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:38.492853 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:38.400649    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:38.804325 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:38.502253    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:38.804325 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:38.602575    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:38.804325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.614940    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 04:58:38.804325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.615090    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 04:58:38.804325 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:38.703953    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:38.804325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.754280    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 04:53:37 +0000 UTC" deadline="2027-12-28 03:14:29.899628765 +0000 UTC"
Jul 15 04:58:38.804325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.754311    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12742h15m51.145321389s"
Jul 15 04:58:38.804325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.800989    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 04:58:38.804325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.802581    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 04:58:38.804325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.802615    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 04:58:38.804325 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.802627    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeHasSufficientPID"
Jul 15 04:58:38.804325 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:38.804234    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:39.088308 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:38.877320    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd9a57e0221e381594f2ffabc783a23f.slice/crio-6f5399e77f8d1dc5a28241fd2fa3b8b1cda2c36f0a06be5caccfbd76a57d1ee5 WatchSource:0}: Error finding container 6f5399e77f8d1dc5a28241fd2fa3b8b1cda2c36f0a06be5caccfbd76a57d1ee5: Status 404 returned error can't find the container with id 6f5399e77f8d1dc5a28241fd2fa3b8b1cda2c36f0a06be5caccfbd76a57d1ee5
Jul 15 04:58:39.088308 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:38.882182    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e99f318c4c76ca78a270f6e363c8e1.slice/crio-43b641ee1f89cb2a00b22c486218c89214abc3f8ed0ef93daa3db22e538cd3ec WatchSource:0}: Error finding container 43b641ee1f89cb2a00b22c486218c89214abc3f8ed0ef93daa3db22e538cd3ec: Status 404 returned error can't find the container with id 43b641ee1f89cb2a00b22c486218c89214abc3f8ed0ef93daa3db22e538cd3ec
Jul 15 04:58:39.088308 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:38.882398    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 04:58:39.088308 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:38.905313    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:39.088308 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:39.007566    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-231.ec2.internal\" not found"
Jul 15 04:58:39.088308 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.055436    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:58:39.088308 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.065607    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:39.088308 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.077727    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 04:58:39.088308 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.078819    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-231.ec2.internal"
Jul 15 04:58:39.090825 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.090721    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 04:58:39.416354 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.130902    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:58:39.416354 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.146430    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:58:39.660092 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.647160    2562 apiserver.go:52] "Watching apiserver"
Jul 15 04:58:39.660092 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.655460    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 04:58:39.660092 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.656387    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5","openshift-cluster-node-tuning-operator/tuned-c8jpr","openshift-dns/node-resolver-4tjhh","openshift-image-registry/node-ca-6tc82","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal","openshift-multus/multus-6rksr","kube-system/konnectivity-agent-gp2nd","openshift-multus/multus-additional-cni-plugins-b46gk","openshift-multus/network-metrics-daemon-9vxv5","openshift-network-diagnostics/network-check-target-v8c4q","openshift-network-operator/iptables-alerter-vq49q","openshift-ovn-kubernetes/ovnkube-node-djb4c","kube-system/global-pull-secret-syncer-tkz8s","kube-system/kube-apiserver-proxy-ip-10-0-128-231.ec2.internal"]
Jul 15 04:58:39.660092 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.659323    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gp2nd"
Jul 15 04:58:39.662867 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.661655    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vq49q"
Jul 15 04:58:39.664242 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.663790    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4tjhh"
Jul 15 04:58:39.666556 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.665822    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 04:58:39.666556 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.666105    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6tc82"
Jul 15 04:58:39.669263 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.668857    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.671791 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.671297    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.673963    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:39.674042    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.675783    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:39.675850    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.677812    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b13f25c6-5faa-4d10-94e9-1174def82690-tmp-dir\") pod \"node-resolver-4tjhh\" (UID: \"b13f25c6-5faa-4d10-94e9-1174def82690\") " pod="openshift-dns/node-resolver-4tjhh"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.677845    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czn7m\" (UniqueName: \"kubernetes.io/projected/b13f25c6-5faa-4d10-94e9-1174def82690-kube-api-access-czn7m\") pod \"node-resolver-4tjhh\" (UID: \"b13f25c6-5faa-4d10-94e9-1174def82690\") " pod="openshift-dns/node-resolver-4tjhh"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.677875    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e3d94ce3-5595-45ad-90e8-b5da6d480cc5-serviceca\") pod \"node-ca-6tc82\" (UID: \"e3d94ce3-5595-45ad-90e8-b5da6d480cc5\") " pod="openshift-image-registry/node-ca-6tc82"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.677899    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2qqm\" (UniqueName: \"kubernetes.io/projected/e3d94ce3-5595-45ad-90e8-b5da6d480cc5-kube-api-access-r2qqm\") pod \"node-ca-6tc82\" (UID: \"e3d94ce3-5595-45ad-90e8-b5da6d480cc5\") " pod="openshift-image-registry/node-ca-6tc82"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.677932    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e49b01bc-0081-4961-90a6-0b2fd594abf7-agent-certs\") pod \"konnectivity-agent-gp2nd\" (UID: \"e49b01bc-0081-4961-90a6-0b2fd594abf7\") " pod="kube-system/konnectivity-agent-gp2nd"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.677978    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e49b01bc-0081-4961-90a6-0b2fd594abf7-konnectivity-ca\") pod \"konnectivity-agent-gp2nd\" (UID: \"e49b01bc-0081-4961-90a6-0b2fd594abf7\") " pod="kube-system/konnectivity-agent-gp2nd"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.678052    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9af2c199-0437-4e1a-ae12-0113b8e325b9-iptables-alerter-script\") pod \"iptables-alerter-vq49q\" (UID: \"9af2c199-0437-4e1a-ae12-0113b8e325b9\") " pod="openshift-network-operator/iptables-alerter-vq49q"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.678094    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9af2c199-0437-4e1a-ae12-0113b8e325b9-host-slash\") pod \"iptables-alerter-vq49q\" (UID: \"9af2c199-0437-4e1a-ae12-0113b8e325b9\") " pod="openshift-network-operator/iptables-alerter-vq49q"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.678121    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm5vj\" (UniqueName: \"kubernetes.io/projected/9af2c199-0437-4e1a-ae12-0113b8e325b9-kube-api-access-nm5vj\") pod \"iptables-alerter-vq49q\" (UID: \"9af2c199-0437-4e1a-ae12-0113b8e325b9\") " pod="openshift-network-operator/iptables-alerter-vq49q"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.678131    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.678160    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3d94ce3-5595-45ad-90e8-b5da6d480cc5-host\") pod \"node-ca-6tc82\" (UID: \"e3d94ce3-5595-45ad-90e8-b5da6d480cc5\") " pod="openshift-image-registry/node-ca-6tc82"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.678188    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b13f25c6-5faa-4d10-94e9-1174def82690-hosts-file\") pod \"node-resolver-4tjhh\" (UID: \"b13f25c6-5faa-4d10-94e9-1174def82690\") " pod="openshift-dns/node-resolver-4tjhh"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.680657    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:39.680717    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:58:39.781789 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.681370    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.681447    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.681627    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-l6gdd\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.681624    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.681654    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.685302    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.687021    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.687437    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wdjp5\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.687516    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.687726    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.687931    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.688309    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.688537    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lvkcd\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.688686    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.690593    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-b8twq\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.691814    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5tz74\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.691952    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.692066    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.692221    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k9fk8\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.701184    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.701375    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-crgbt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.701669    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.702036    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.702156    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.702419    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.702583    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.704482    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.707687    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.707830    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.708228    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.708369    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fbn89\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.722791    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.722817    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.722874    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.722870    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.723779    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.724513    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 04:58:39.783148 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.724751    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4szt8\""
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.729705    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.729831    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fcfg5"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.767056    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.778925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9af2c199-0437-4e1a-ae12-0113b8e325b9-iptables-alerter-script\") pod \"iptables-alerter-vq49q\" (UID: \"9af2c199-0437-4e1a-ae12-0113b8e325b9\") " pod="openshift-network-operator/iptables-alerter-vq49q"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.778982    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-var-lib-kubelet\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779013    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlgpg\" (UniqueName: \"kubernetes.io/projected/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-kube-api-access-mlgpg\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779059    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-kubelet\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-ovnkube-config\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779108    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-env-overrides\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779150    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-run-netns\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779173    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-var-lib-cni-multus\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779190    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/674d12b8-5912-4f32-b965-1a498763f81c-multus-daemon-config\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779226    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btsqc\" (UniqueName: \"kubernetes.io/projected/30b62c6b-b454-403b-b1be-4fd921f4a5e4-kube-api-access-btsqc\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779251    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-sysctl-conf\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779279    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-ovnkube-script-lib\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779307    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2qqm\" (UniqueName: \"kubernetes.io/projected/e3d94ce3-5595-45ad-90e8-b5da6d480cc5-kube-api-access-r2qqm\") pod \"node-ca-6tc82\" (UID: \"e3d94ce3-5595-45ad-90e8-b5da6d480cc5\") " pod="openshift-image-registry/node-ca-6tc82"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779332    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e49b01bc-0081-4961-90a6-0b2fd594abf7-konnectivity-ca\") pod \"konnectivity-agent-gp2nd\" (UID: \"e49b01bc-0081-4961-90a6-0b2fd594abf7\") " pod="kube-system/konnectivity-agent-gp2nd"
Jul 15 04:58:39.785461 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779367    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7dced01d-9de8-4e21-8ca3-241d08ecab2e-kubelet-config\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779417    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-sys-fs\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779460    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779493    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-system-cni-dir\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779517    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-cnibin\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779555    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-os-release\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-cni-binary-copy\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779610    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-run-multus-certs\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779633    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-run-netns\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779658    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-ovn-node-metrics-cert\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779685    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e3d94ce3-5595-45ad-90e8-b5da6d480cc5-serviceca\") pod \"node-ca-6tc82\" (UID: \"e3d94ce3-5595-45ad-90e8-b5da6d480cc5\") " pod="openshift-image-registry/node-ca-6tc82"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779723    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e49b01bc-0081-4961-90a6-0b2fd594abf7-agent-certs\") pod \"konnectivity-agent-gp2nd\" (UID: \"e49b01bc-0081-4961-90a6-0b2fd594abf7\") " pod="kube-system/konnectivity-agent-gp2nd"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779750    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779776    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-cnibin\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779799    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-tmp\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779823    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-etc-openvswitch\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.787189 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3d94ce3-5595-45ad-90e8-b5da6d480cc5-host\") pod \"node-ca-6tc82\" (UID: \"e3d94ce3-5595-45ad-90e8-b5da6d480cc5\") " pod="openshift-image-registry/node-ca-6tc82"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779870    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czn7m\" (UniqueName: \"kubernetes.io/projected/b13f25c6-5faa-4d10-94e9-1174def82690-kube-api-access-czn7m\") pod \"node-resolver-4tjhh\" (UID: \"b13f25c6-5faa-4d10-94e9-1174def82690\") " pod="openshift-dns/node-resolver-4tjhh"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779890    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-multus-conf-dir\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779914    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-host\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779945    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9af2c199-0437-4e1a-ae12-0113b8e325b9-host-slash\") pod \"iptables-alerter-vq49q\" (UID: \"9af2c199-0437-4e1a-ae12-0113b8e325b9\") " pod="openshift-network-operator/iptables-alerter-vq49q"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.779977    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/674d12b8-5912-4f32-b965-1a498763f81c-cni-binary-copy\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-socket-dir\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780033    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-device-dir\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780058    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-etc-selinux\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-slash\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780108    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-run-ovn\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780162    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-run-systemd\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780188    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-hostroot\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780213    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-sysconfig\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780239    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-sys\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-var-lib-kubelet\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.788205 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780298    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-tuned\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780326    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780354    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-cni-bin\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780380    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-multus-cni-dir\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8m9g\" (UniqueName: \"kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g\") pod \"network-check-target-v8c4q\" (UID: \"8bd2c88a-1300-41b1-984f-fc4d17ff45f6\") " pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780437    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-kubernetes\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780476    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-run\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780503    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-registration-dir\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780529    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwnqw\" (UniqueName: \"kubernetes.io/projected/674d12b8-5912-4f32-b965-1a498763f81c-kube-api-access-bwnqw\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780553    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-modprobe-d\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780592    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-sysctl-d\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780609    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-lib-modules\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780629    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-systemd\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780658    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-run-k8s-cni-cncf-io\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780678    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-etc-kubernetes\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780695    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gr4f\" (UniqueName: \"kubernetes.io/projected/3d04b791-6db8-4b54-9670-cc198c1868af-kube-api-access-2gr4f\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.790261 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780720    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-run-ovn-kubernetes\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780747    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-cni-netd\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b13f25c6-5faa-4d10-94e9-1174def82690-tmp-dir\") pod \"node-resolver-4tjhh\" (UID: \"b13f25c6-5faa-4d10-94e9-1174def82690\") " pod="openshift-dns/node-resolver-4tjhh"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm5vj\" (UniqueName: \"kubernetes.io/projected/9af2c199-0437-4e1a-ae12-0113b8e325b9-kube-api-access-nm5vj\") pod \"iptables-alerter-vq49q\" (UID: \"9af2c199-0437-4e1a-ae12-0113b8e325b9\") " pod="openshift-network-operator/iptables-alerter-vq49q"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780835    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780865    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780893    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b13f25c6-5faa-4d10-94e9-1174def82690-hosts-file\") pod \"node-resolver-4tjhh\" (UID: \"b13f25c6-5faa-4d10-94e9-1174def82690\") " pod="openshift-dns/node-resolver-4tjhh"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780922    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5l57\" (UniqueName: \"kubernetes.io/projected/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-kube-api-access-m5l57\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780949    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-system-cni-dir\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.780976    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7dced01d-9de8-4e21-8ca3-241d08ecab2e-dbus\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.781003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-systemd-units\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.781030    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-log-socket\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.781058    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdfcw\" (UniqueName: \"kubernetes.io/projected/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-kube-api-access-kdfcw\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.781086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.781115    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-var-lib-cni-bin\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.781152    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-run-openvswitch\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.791645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.781184    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-multus-socket-dir-parent\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.793526 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.781219    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-var-lib-openvswitch\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.793526 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.781247    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-node-log\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.793526 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.781274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-os-release\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.793526 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.781935    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9af2c199-0437-4e1a-ae12-0113b8e325b9-iptables-alerter-script\") pod \"iptables-alerter-vq49q\" (UID: \"9af2c199-0437-4e1a-ae12-0113b8e325b9\") " pod="openshift-network-operator/iptables-alerter-vq49q"
Jul 15 04:58:39.793526 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.782779    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e49b01bc-0081-4961-90a6-0b2fd594abf7-konnectivity-ca\") pod \"konnectivity-agent-gp2nd\" (UID: \"e49b01bc-0081-4961-90a6-0b2fd594abf7\") " pod="kube-system/konnectivity-agent-gp2nd"
Jul 15 04:58:39.793526 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.783265    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e3d94ce3-5595-45ad-90e8-b5da6d480cc5-serviceca\") pod \"node-ca-6tc82\" (UID: \"e3d94ce3-5595-45ad-90e8-b5da6d480cc5\") " pod="openshift-image-registry/node-ca-6tc82"
Jul 15 04:58:39.793526 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.783607    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 04:58:39.793526 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.784002    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3d94ce3-5595-45ad-90e8-b5da6d480cc5-host\") pod \"node-ca-6tc82\" (UID: \"e3d94ce3-5595-45ad-90e8-b5da6d480cc5\") " pod="openshift-image-registry/node-ca-6tc82"
Jul 15 04:58:39.793526 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.784208    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9af2c199-0437-4e1a-ae12-0113b8e325b9-host-slash\") pod \"iptables-alerter-vq49q\" (UID: \"9af2c199-0437-4e1a-ae12-0113b8e325b9\") " pod="openshift-network-operator/iptables-alerter-vq49q"
Jul 15 04:58:39.793526 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.784606    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b13f25c6-5faa-4d10-94e9-1174def82690-tmp-dir\") pod \"node-resolver-4tjhh\" (UID: \"b13f25c6-5faa-4d10-94e9-1174def82690\") " pod="openshift-dns/node-resolver-4tjhh"
Jul 15 04:58:39.793526 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.785042    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b13f25c6-5faa-4d10-94e9-1174def82690-hosts-file\") pod \"node-resolver-4tjhh\" (UID: \"b13f25c6-5faa-4d10-94e9-1174def82690\") " pod="openshift-dns/node-resolver-4tjhh"
Jul 15 04:58:39.793526 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.787685    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e49b01bc-0081-4961-90a6-0b2fd594abf7-agent-certs\") pod \"konnectivity-agent-gp2nd\" (UID: \"e49b01bc-0081-4961-90a6-0b2fd594abf7\") " pod="kube-system/konnectivity-agent-gp2nd"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.807794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm5vj\" (UniqueName: \"kubernetes.io/projected/9af2c199-0437-4e1a-ae12-0113b8e325b9-kube-api-access-nm5vj\") pod \"iptables-alerter-vq49q\" (UID: \"9af2c199-0437-4e1a-ae12-0113b8e325b9\") " pod="openshift-network-operator/iptables-alerter-vq49q"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.810188    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czn7m\" (UniqueName: \"kubernetes.io/projected/b13f25c6-5faa-4d10-94e9-1174def82690-kube-api-access-czn7m\") pod \"node-resolver-4tjhh\" (UID: \"b13f25c6-5faa-4d10-94e9-1174def82690\") " pod="openshift-dns/node-resolver-4tjhh"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.812211    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2qqm\" (UniqueName: \"kubernetes.io/projected/e3d94ce3-5595-45ad-90e8-b5da6d480cc5-kube-api-access-r2qqm\") pod \"node-ca-6tc82\" (UID: \"e3d94ce3-5595-45ad-90e8-b5da6d480cc5\") " pod="openshift-image-registry/node-ca-6tc82"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.844636    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-231.ec2.internal" event={"ID":"33e99f318c4c76ca78a270f6e363c8e1","Type":"ContainerStarted","Data":"43b641ee1f89cb2a00b22c486218c89214abc3f8ed0ef93daa3db22e538cd3ec"}
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.846030    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal" event={"ID":"cd9a57e0221e381594f2ffabc783a23f","Type":"ContainerStarted","Data":"6f5399e77f8d1dc5a28241fd2fa3b8b1cda2c36f0a06be5caccfbd76a57d1ee5"}
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.883056    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-var-lib-cni-multus\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.883094    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/674d12b8-5912-4f32-b965-1a498763f81c-multus-daemon-config\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.883338    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-var-lib-cni-multus\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.883253    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btsqc\" (UniqueName: \"kubernetes.io/projected/30b62c6b-b454-403b-b1be-4fd921f4a5e4-kube-api-access-btsqc\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.883519    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-sysctl-conf\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.883544    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-ovnkube-script-lib\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.883572    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7dced01d-9de8-4e21-8ca3-241d08ecab2e-kubelet-config\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.883685    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-sys-fs\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.883708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.883797    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-system-cni-dir\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.883923    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-cnibin\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.918234 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.883961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-os-release\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.883983    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-cni-binary-copy\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884007    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-run-multus-certs\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884019    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7dced01d-9de8-4e21-8ca3-241d08ecab2e-kubelet-config\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884029    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-run-netns\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884065    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-ovn-node-metrics-cert\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884072    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-run-netns\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884099    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-cnibin\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884131    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-sys-fs\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884152    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-tmp\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884177    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-etc-openvswitch\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884176    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884208    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-multus-conf-dir\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-host\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884236    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-system-cni-dir\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884262    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/674d12b8-5912-4f32-b965-1a498763f81c-cni-binary-copy\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.920304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884289    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-socket-dir\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884309    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-cnibin\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884318    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-device-dir\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884346    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-etc-selinux\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884371    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-slash\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884378    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-os-release\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884403    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-run-ovn\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884433    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884481    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-cnibin\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884557    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-run-systemd\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884602    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-hostroot\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884628    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-sysconfig\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884649    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-sys\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884671    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-var-lib-kubelet\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884705    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-tuned\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884749    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884780    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-cni-bin\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-multus-cni-dir\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.921400 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884834    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-sysctl-conf\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884842    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8m9g\" (UniqueName: \"kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g\") pod \"network-check-target-v8c4q\" (UID: \"8bd2c88a-1300-41b1-984f-fc4d17ff45f6\") " pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884906    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-kubernetes\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884942    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-run\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-registration-dir\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885005    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwnqw\" (UniqueName: \"kubernetes.io/projected/674d12b8-5912-4f32-b965-1a498763f81c-kube-api-access-bwnqw\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885033    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-modprobe-d\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885059    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-sysctl-d\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885085    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-lib-modules\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-systemd\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885136    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-run-k8s-cni-cncf-io\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885162    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-etc-kubernetes\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885189    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gr4f\" (UniqueName: \"kubernetes.io/projected/3d04b791-6db8-4b54-9670-cc198c1868af-kube-api-access-2gr4f\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885219    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-run-ovn-kubernetes\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885238    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-cni-binary-copy\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885251    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-cni-netd\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885293    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-cni-netd\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.922294 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885340    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885343    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-kubernetes\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885389    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-run\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.885398    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-run-multus-certs\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.887900    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-run-systemd\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:39.887976    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.888005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-hostroot\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.888076    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-sysconfig\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:39.888102    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs podName:30b62c6b-b454-403b-b1be-4fd921f4a5e4 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:40.388067637 +0000 UTC m=+2.991620249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs") pod "network-metrics-daemon-9vxv5" (UID: "30b62c6b-b454-403b-b1be-4fd921f4a5e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.888145    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.888162    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-sys\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.888228    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-var-lib-kubelet\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.888399    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-tmp\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.888490    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.888532    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-systemd\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.888586    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-cni-bin\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.923047 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.888609    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-registration-dir\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.888742    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-multus-cni-dir\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889067    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-modprobe-d\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889072    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-run-k8s-cni-cncf-io\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889121    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-etc-kubernetes\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889210    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-etc-openvswitch\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889226    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-sysctl-d\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889294    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-multus-conf-dir\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-lib-modules\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.884318    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/674d12b8-5912-4f32-b965-1a498763f81c-multus-daemon-config\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889380    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-host\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889407    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-run-ovn-kubernetes\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889475    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-socket-dir\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889476    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5l57\" (UniqueName: \"kubernetes.io/projected/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-kube-api-access-m5l57\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889520    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-system-cni-dir\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889561    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-device-dir\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889689    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7dced01d-9de8-4e21-8ca3-241d08ecab2e-dbus\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889722    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-systemd-units\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.923813 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889750    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-log-socket\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889782    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-slash\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.889941    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-run-ovn\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890027    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdfcw\" (UniqueName: \"kubernetes.io/projected/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-kube-api-access-kdfcw\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890053    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-log-socket\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890071    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890105    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-var-lib-cni-bin\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890113    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-system-cni-dir\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-run-openvswitch\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890281    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-multus-socket-dir-parent\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890346    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-var-lib-openvswitch\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890431    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/674d12b8-5912-4f32-b965-1a498763f81c-cni-binary-copy\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:39.890470    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890505    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3d04b791-6db8-4b54-9670-cc198c1868af-etc-selinux\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:39.890552    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret podName:7dced01d-9de8-4e21-8ca3-241d08ecab2e nodeName:}" failed. No retries permitted until 2026-07-15 04:58:40.390532388 +0000 UTC m=+2.994084991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret") pod "global-pull-secret-syncer-tkz8s" (UID: "7dced01d-9de8-4e21-8ca3-241d08ecab2e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890592    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890612    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-run-openvswitch\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.924626 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-node-log\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890694    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-systemd-units\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890723    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-os-release\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890743    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-var-lib-cni-bin\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890775    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-var-lib-openvswitch\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890792    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7dced01d-9de8-4e21-8ca3-241d08ecab2e-dbus\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.890826    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-node-log\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.891191    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-os-release\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.891289    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-var-lib-kubelet\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.891328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlgpg\" (UniqueName: \"kubernetes.io/projected/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-kube-api-access-mlgpg\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.891377    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.891391    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-kubelet\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.891441    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-ovnkube-config\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.891495    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-env-overrides\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.891541    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-run-netns\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:39.891666    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:39.891690    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:39.891705    2562 projected.go:196] Error preparing data for projected volume kube-api-access-r8m9g for pod openshift-network-diagnostics/network-check-target-v8c4q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:39.925364 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.891752    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-run-netns\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:39.891782    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g podName:8bd2c88a-1300-41b1-984f-fc4d17ff45f6 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:40.39176298 +0000 UTC m=+2.995315572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r8m9g" (UniqueName: "kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g") pod "network-check-target-v8c4q" (UID: "8bd2c88a-1300-41b1-984f-fc4d17ff45f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.891849    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-host-kubelet\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.891913    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-multus-socket-dir-parent\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.891960    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/674d12b8-5912-4f32-b965-1a498763f81c-host-var-lib-kubelet\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.892427    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-ovnkube-config\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.892557    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-env-overrides\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.892639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-ovnkube-script-lib\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.894770    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-etc-tuned\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.896045    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-ovn-node-metrics-cert\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.897167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btsqc\" (UniqueName: \"kubernetes.io/projected/30b62c6b-b454-403b-b1be-4fd921f4a5e4-kube-api-access-btsqc\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.899055    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5l57\" (UniqueName: \"kubernetes.io/projected/24d3a875-2a2f-47a9-9842-c12d04ad9fb9-kube-api-access-m5l57\") pod \"multus-additional-cni-plugins-b46gk\" (UID: \"24d3a875-2a2f-47a9-9842-c12d04ad9fb9\") " pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.899630    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdfcw\" (UniqueName: \"kubernetes.io/projected/e1aacd1d-1083-4b6d-8eb6-064cb6be4430-kube-api-access-kdfcw\") pod \"ovnkube-node-djb4c\" (UID: \"e1aacd1d-1083-4b6d-8eb6-064cb6be4430\") " pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.899954    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwnqw\" (UniqueName: \"kubernetes.io/projected/674d12b8-5912-4f32-b965-1a498763f81c-kube-api-access-bwnqw\") pod \"multus-6rksr\" (UID: \"674d12b8-5912-4f32-b965-1a498763f81c\") " pod="openshift-multus/multus-6rksr"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.900577    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlgpg\" (UniqueName: \"kubernetes.io/projected/f0bc6afb-365d-4ab0-8088-a67c2c9a4316-kube-api-access-mlgpg\") pod \"tuned-c8jpr\" (UID: \"f0bc6afb-365d-4ab0-8088-a67c2c9a4316\") " pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:39.926295 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.911038    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gr4f\" (UniqueName: \"kubernetes.io/projected/3d04b791-6db8-4b54-9670-cc198c1868af-kube-api-access-2gr4f\") pod \"aws-ebs-csi-driver-node-fd7t5\" (UID: \"3d04b791-6db8-4b54-9670-cc198c1868af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:40.132694 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.972363    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gp2nd"
Jul 15 04:58:40.132694 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.986995    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vq49q"
Jul 15 04:58:40.132694 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:39.988192    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 04:58:40.132694 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:40.005459    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4tjhh"
Jul 15 04:58:40.132694 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:40.026099    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6tc82"
Jul 15 04:58:40.132694 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:40.036384    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b46gk"
Jul 15 04:58:40.132694 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:40.044156    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6rksr"
Jul 15 04:58:40.132694 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:40.052972    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c8jpr"
Jul 15 04:58:40.132694 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:40.062737    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5"
Jul 15 04:58:40.132694 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:40.067515    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:58:40.482524 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:40.394517    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:40.482524 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:40.394562    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:40.482524 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:40.394628    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8m9g\" (UniqueName: \"kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g\") pod \"network-check-target-v8c4q\" (UID: \"8bd2c88a-1300-41b1-984f-fc4d17ff45f6\") " pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:40.482524 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:40.394679    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:40.482524 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:40.394744    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:40.482524 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:40.394749    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs podName:30b62c6b-b454-403b-b1be-4fd921f4a5e4 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:41.394726498 +0000 UTC m=+3.998279087 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs") pod "network-metrics-daemon-9vxv5" (UID: "30b62c6b-b454-403b-b1be-4fd921f4a5e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:40.482524 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:40.394796    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:58:40.482524 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:40.394811    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:58:40.482524 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:40.394813    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret podName:7dced01d-9de8-4e21-8ca3-241d08ecab2e nodeName:}" failed. No retries permitted until 2026-07-15 04:58:41.394796 +0000 UTC m=+3.998348599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret") pod "global-pull-secret-syncer-tkz8s" (UID: "7dced01d-9de8-4e21-8ca3-241d08ecab2e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:40.482524 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:40.394824    2562 projected.go:196] Error preparing data for projected volume kube-api-access-r8m9g for pod openshift-network-diagnostics/network-check-target-v8c4q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:40.482524 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:40.394872    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g podName:8bd2c88a-1300-41b1-984f-fc4d17ff45f6 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:41.39485902 +0000 UTC m=+3.998411612 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8m9g" (UniqueName: "kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g") pod "network-check-target-v8c4q" (UID: "8bd2c88a-1300-41b1-984f-fc4d17ff45f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:40.951922 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:40.735057    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 04:53:39 +0000 UTC" deadline="2028-02-22 15:02:52.390004088 +0000 UTC"
Jul 15 04:58:40.951922 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:40.735090    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14098h4m11.654917323s"
Jul 15 04:58:41.024815 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:41.024771    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1aacd1d_1083_4b6d_8eb6_064cb6be4430.slice/crio-8021955f58ef4822dab1fa74ed82afbee245efdadf38912b7ba2c0027190c040 WatchSource:0}: Error finding container 8021955f58ef4822dab1fa74ed82afbee245efdadf38912b7ba2c0027190c040: Status 404 returned error can't find the container with id 8021955f58ef4822dab1fa74ed82afbee245efdadf38912b7ba2c0027190c040
Jul 15 04:58:41.050833 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:41.050787    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9af2c199_0437_4e1a_ae12_0113b8e325b9.slice/crio-11bbcd56eab430e47f228495d63353a952843cc7f61db44a89740bffdc3d65ef WatchSource:0}: Error finding container 11bbcd56eab430e47f228495d63353a952843cc7f61db44a89740bffdc3d65ef: Status 404 returned error can't find the container with id 11bbcd56eab430e47f228495d63353a952843cc7f61db44a89740bffdc3d65ef
Jul 15 04:58:41.052003 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:41.051972    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod674d12b8_5912_4f32_b965_1a498763f81c.slice/crio-ad9350ec353a71ab37a0f08dbd20b21312a8db255b73b9cdcbb34a078fe90819 WatchSource:0}: Error finding container ad9350ec353a71ab37a0f08dbd20b21312a8db255b73b9cdcbb34a078fe90819: Status 404 returned error can't find the container with id ad9350ec353a71ab37a0f08dbd20b21312a8db255b73b9cdcbb34a078fe90819
Jul 15 04:58:41.052978 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:41.052942    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode49b01bc_0081_4961_90a6_0b2fd594abf7.slice/crio-b210b9aec42a53910e5d1179d1cb48e74f3ecab212083c66910ed8e586e4aae9 WatchSource:0}: Error finding container b210b9aec42a53910e5d1179d1cb48e74f3ecab212083c66910ed8e586e4aae9: Status 404 returned error can't find the container with id b210b9aec42a53910e5d1179d1cb48e74f3ecab212083c66910ed8e586e4aae9
Jul 15 04:58:41.053851 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:41.053731    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d04b791_6db8_4b54_9670_cc198c1868af.slice/crio-099e2b1279eb9cc402d3adec1e6ae80c1652d574e20caaa4cf1ebff9d4504519 WatchSource:0}: Error finding container 099e2b1279eb9cc402d3adec1e6ae80c1652d574e20caaa4cf1ebff9d4504519: Status 404 returned error can't find the container with id 099e2b1279eb9cc402d3adec1e6ae80c1652d574e20caaa4cf1ebff9d4504519
Jul 15 04:58:41.054543 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:41.054477    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d3a875_2a2f_47a9_9842_c12d04ad9fb9.slice/crio-f669a16998a790b17986dfd5f4a9d2114a06005aa741074b36c703c2851943d7 WatchSource:0}: Error finding container f669a16998a790b17986dfd5f4a9d2114a06005aa741074b36c703c2851943d7: Status 404 returned error can't find the container with id f669a16998a790b17986dfd5f4a9d2114a06005aa741074b36c703c2851943d7
Jul 15 04:58:41.055774 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:41.055752    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0bc6afb_365d_4ab0_8088_a67c2c9a4316.slice/crio-07946240e6d49ba246ae2085dd011d666a51787b4c7819700534f06b28c8e3bf WatchSource:0}: Error finding container 07946240e6d49ba246ae2085dd011d666a51787b4c7819700534f06b28c8e3bf: Status 404 returned error can't find the container with id 07946240e6d49ba246ae2085dd011d666a51787b4c7819700534f06b28c8e3bf
Jul 15 04:58:41.057180 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:58:41.056637    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d94ce3_5595_45ad_90e8_b5da6d480cc5.slice/crio-b587e5dc380066cc5135e43b5096184da03a06769cacfe021b52a5a9f6b3cf8a WatchSource:0}: Error finding container b587e5dc380066cc5135e43b5096184da03a06769cacfe021b52a5a9f6b3cf8a: Status 404 returned error can't find the container with id b587e5dc380066cc5135e43b5096184da03a06769cacfe021b52a5a9f6b3cf8a
Jul 15 04:58:41.406199 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.405849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8m9g\" (UniqueName: \"kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g\") pod \"network-check-target-v8c4q\" (UID: \"8bd2c88a-1300-41b1-984f-fc4d17ff45f6\") " pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:41.406199 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.406161    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:41.406199 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.406193    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:41.406494 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:41.406044    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:58:41.406494 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:41.406351    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:58:41.406494 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:41.406379    2562 projected.go:196] Error preparing data for projected volume kube-api-access-r8m9g for pod openshift-network-diagnostics/network-check-target-v8c4q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:41.406494 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:41.406441    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g podName:8bd2c88a-1300-41b1-984f-fc4d17ff45f6 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:43.406421854 +0000 UTC m=+6.009974460 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8m9g" (UniqueName: "kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g") pod "network-check-target-v8c4q" (UID: "8bd2c88a-1300-41b1-984f-fc4d17ff45f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:41.406811 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:41.406323    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:41.406811 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:41.406781    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret podName:7dced01d-9de8-4e21-8ca3-241d08ecab2e nodeName:}" failed. No retries permitted until 2026-07-15 04:58:43.406760998 +0000 UTC m=+6.010313587 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret") pod "global-pull-secret-syncer-tkz8s" (UID: "7dced01d-9de8-4e21-8ca3-241d08ecab2e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:41.406929 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:41.406842    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:41.406929 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:41.406894    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs podName:30b62c6b-b454-403b-b1be-4fd921f4a5e4 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:43.406880642 +0000 UTC m=+6.010433232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs") pod "network-metrics-daemon-9vxv5" (UID: "30b62c6b-b454-403b-b1be-4fd921f4a5e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:41.736073 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.735959    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 04:53:39 +0000 UTC" deadline="2028-01-26 17:10:21.372574256 +0000 UTC"
Jul 15 04:58:41.736073 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.736023    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13452h11m39.636555707s"
Jul 15 04:58:41.841710 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.841679    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:41.841876 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:41.841821    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:58:41.842230 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.842208    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:41.842345 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:41.842324    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:58:41.842416 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.842408    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:41.842507 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:41.842485    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:58:41.865453 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.865345    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-231.ec2.internal" event={"ID":"33e99f318c4c76ca78a270f6e363c8e1","Type":"ContainerStarted","Data":"3f1a188e3688bdd1eddfbedc38b0e06528eee402edab99921fb2a1de014840d2"}
Jul 15 04:58:41.877243 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.877105    2562 generic.go:356] "Generic (PLEG): container finished" podID="cd9a57e0221e381594f2ffabc783a23f" containerID="3ee29f34b1a68b34596946b0bec92c434b5550f5463c7e3a3ed606dc233ea1e8" exitCode=0
Jul 15 04:58:41.877243 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.877195    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal" event={"ID":"cd9a57e0221e381594f2ffabc783a23f","Type":"ContainerDied","Data":"3ee29f34b1a68b34596946b0bec92c434b5550f5463c7e3a3ed606dc233ea1e8"}
Jul 15 04:58:41.884120 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.883416    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c8jpr" event={"ID":"f0bc6afb-365d-4ab0-8088-a67c2c9a4316","Type":"ContainerStarted","Data":"07946240e6d49ba246ae2085dd011d666a51787b4c7819700534f06b28c8e3bf"}
Jul 15 04:58:41.894611 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.894545    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-231.ec2.internal" podStartSLOduration=2.8945274359999997 podStartE2EDuration="2.894527436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:58:41.880350941 +0000 UTC m=+4.483903549" watchObservedRunningTime="2026-07-15 04:58:41.894527436 +0000 UTC m=+4.498080047"
Jul 15 04:58:41.897779 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.897751    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b46gk" event={"ID":"24d3a875-2a2f-47a9-9842-c12d04ad9fb9","Type":"ContainerStarted","Data":"f669a16998a790b17986dfd5f4a9d2114a06005aa741074b36c703c2851943d7"}
Jul 15 04:58:41.902944 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.902916    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gp2nd" event={"ID":"e49b01bc-0081-4961-90a6-0b2fd594abf7","Type":"ContainerStarted","Data":"b210b9aec42a53910e5d1179d1cb48e74f3ecab212083c66910ed8e586e4aae9"}
Jul 15 04:58:41.918621 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.918586    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4tjhh" event={"ID":"b13f25c6-5faa-4d10-94e9-1174def82690","Type":"ContainerStarted","Data":"61b1b6cf468569d86a2c467b0383660b9a4abea81be489cc44f1ebb5edcac9d0"}
Jul 15 04:58:41.934587 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.934558    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6tc82" event={"ID":"e3d94ce3-5595-45ad-90e8-b5da6d480cc5","Type":"ContainerStarted","Data":"b587e5dc380066cc5135e43b5096184da03a06769cacfe021b52a5a9f6b3cf8a"}
Jul 15 04:58:41.948672 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.948635    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5" event={"ID":"3d04b791-6db8-4b54-9670-cc198c1868af","Type":"ContainerStarted","Data":"099e2b1279eb9cc402d3adec1e6ae80c1652d574e20caaa4cf1ebff9d4504519"}
Jul 15 04:58:41.957680 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.957649    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6rksr" event={"ID":"674d12b8-5912-4f32-b965-1a498763f81c","Type":"ContainerStarted","Data":"ad9350ec353a71ab37a0f08dbd20b21312a8db255b73b9cdcbb34a078fe90819"}
Jul 15 04:58:41.972960 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.972926    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vq49q" event={"ID":"9af2c199-0437-4e1a-ae12-0113b8e325b9","Type":"ContainerStarted","Data":"11bbcd56eab430e47f228495d63353a952843cc7f61db44a89740bffdc3d65ef"}
Jul 15 04:58:41.989518 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:41.989331    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c" event={"ID":"e1aacd1d-1083-4b6d-8eb6-064cb6be4430","Type":"ContainerStarted","Data":"8021955f58ef4822dab1fa74ed82afbee245efdadf38912b7ba2c0027190c040"}
Jul 15 04:58:43.014422 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:43.014352    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal" event={"ID":"cd9a57e0221e381594f2ffabc783a23f","Type":"ContainerStarted","Data":"e87ead677f0d8ae8ea44bb004c8024336e6e6259482ab3c8d5889bd8145cfd47"}
Jul 15 04:58:43.425960 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:43.425873    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8m9g\" (UniqueName: \"kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g\") pod \"network-check-target-v8c4q\" (UID: \"8bd2c88a-1300-41b1-984f-fc4d17ff45f6\") " pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:43.425960 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:43.425935    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:43.425960 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:43.425959    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:43.426215 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:43.426084    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:43.426215 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:43.426145    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret podName:7dced01d-9de8-4e21-8ca3-241d08ecab2e nodeName:}" failed. No retries permitted until 2026-07-15 04:58:47.426126058 +0000 UTC m=+10.029678657 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret") pod "global-pull-secret-syncer-tkz8s" (UID: "7dced01d-9de8-4e21-8ca3-241d08ecab2e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:43.426606 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:43.426585    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:58:43.426704 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:43.426613    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:58:43.426704 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:43.426625    2562 projected.go:196] Error preparing data for projected volume kube-api-access-r8m9g for pod openshift-network-diagnostics/network-check-target-v8c4q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:43.426704 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:43.426671    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g podName:8bd2c88a-1300-41b1-984f-fc4d17ff45f6 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:47.426656084 +0000 UTC m=+10.030208687 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8m9g" (UniqueName: "kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g") pod "network-check-target-v8c4q" (UID: "8bd2c88a-1300-41b1-984f-fc4d17ff45f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:43.426864 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:43.426753    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:43.426864 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:43.426786    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs podName:30b62c6b-b454-403b-b1be-4fd921f4a5e4 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:47.426775156 +0000 UTC m=+10.030327743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs") pod "network-metrics-daemon-9vxv5" (UID: "30b62c6b-b454-403b-b1be-4fd921f4a5e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:43.841195 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:43.840470    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:43.841195 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:43.840611    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:58:43.841195 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:43.840989    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:43.841195 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:43.841085    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:58:43.851041 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:43.847405    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:43.851041 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:43.847554    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:58:45.842297 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:45.841506    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:45.842297 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:45.841636    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:58:45.842297 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:45.842020    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:45.842297 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:45.842123    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:58:45.842297 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:45.842174    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:45.842297 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:45.842243    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:58:47.458265 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:47.458227    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8m9g\" (UniqueName: \"kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g\") pod \"network-check-target-v8c4q\" (UID: \"8bd2c88a-1300-41b1-984f-fc4d17ff45f6\") " pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:47.458732 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:47.458282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:47.458732 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:47.458307    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:47.458732 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:47.458460    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:47.458732 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:47.458522    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret podName:7dced01d-9de8-4e21-8ca3-241d08ecab2e nodeName:}" failed. No retries permitted until 2026-07-15 04:58:55.458503432 +0000 UTC m=+18.062056024 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret") pod "global-pull-secret-syncer-tkz8s" (UID: "7dced01d-9de8-4e21-8ca3-241d08ecab2e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:47.458965 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:47.458931    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:58:47.458965 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:47.458954    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:58:47.458965 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:47.458966    2562 projected.go:196] Error preparing data for projected volume kube-api-access-r8m9g for pod openshift-network-diagnostics/network-check-target-v8c4q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:47.459108 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:47.459015    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g podName:8bd2c88a-1300-41b1-984f-fc4d17ff45f6 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:55.458998196 +0000 UTC m=+18.062550802 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8m9g" (UniqueName: "kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g") pod "network-check-target-v8c4q" (UID: "8bd2c88a-1300-41b1-984f-fc4d17ff45f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:47.459108 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:47.459077    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:47.459108 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:47.459108    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs podName:30b62c6b-b454-403b-b1be-4fd921f4a5e4 nodeName:}" failed. No retries permitted until 2026-07-15 04:58:55.45909813 +0000 UTC m=+18.062650730 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs") pod "network-metrics-daemon-9vxv5" (UID: "30b62c6b-b454-403b-b1be-4fd921f4a5e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:47.841089 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:47.840545    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:47.841089 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:47.840652    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:58:47.841089 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:47.840959    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:47.841089 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:47.841078    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:47.841412 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:47.841164    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:58:47.841412 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:47.841205    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:58:49.839666 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:49.839628    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:49.840089 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:49.839628    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:49.840089 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:49.839754    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:58:49.840089 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:49.839771    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:49.840089 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:49.839857    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:58:49.840089 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:49.839955    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:58:51.840005 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:51.839471    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:51.840005 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:51.839495    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:51.840005 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:51.839540    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:51.840005 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:51.839659    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:58:51.840005 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:51.839799    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:58:51.840005 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:51.839878    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:58:53.840153 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:53.840113    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:53.840153 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:53.840154    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:53.840695 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:53.840114    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:53.840695 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:53.840229    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:58:53.840695 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:53.840301    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:58:53.840695 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:53.840396    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:58:55.520647 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:55.520617    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:55.520647 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:55.520651    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:55.521118 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:55.520685    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8m9g\" (UniqueName: \"kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g\") pod \"network-check-target-v8c4q\" (UID: \"8bd2c88a-1300-41b1-984f-fc4d17ff45f6\") " pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:55.521118 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:55.520774    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:55.521118 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:55.520783    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:58:55.521118 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:55.520783    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:55.521118 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:55.520796    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:58:55.521118 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:55.520807    2562 projected.go:196] Error preparing data for projected volume kube-api-access-r8m9g for pod openshift-network-diagnostics/network-check-target-v8c4q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:55.521118 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:55.520829    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret podName:7dced01d-9de8-4e21-8ca3-241d08ecab2e nodeName:}" failed. No retries permitted until 2026-07-15 04:59:11.520815417 +0000 UTC m=+34.124368004 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret") pod "global-pull-secret-syncer-tkz8s" (UID: "7dced01d-9de8-4e21-8ca3-241d08ecab2e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:58:55.521118 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:55.520845    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs podName:30b62c6b-b454-403b-b1be-4fd921f4a5e4 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:11.52083814 +0000 UTC m=+34.124390726 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs") pod "network-metrics-daemon-9vxv5" (UID: "30b62c6b-b454-403b-b1be-4fd921f4a5e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:58:55.521118 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:55.520855    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g podName:8bd2c88a-1300-41b1-984f-fc4d17ff45f6 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:11.52084945 +0000 UTC m=+34.124402036 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8m9g" (UniqueName: "kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g") pod "network-check-target-v8c4q" (UID: "8bd2c88a-1300-41b1-984f-fc4d17ff45f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:58:55.839352 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:55.839326    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:55.839474 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:55.839330    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:55.839474 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:55.839365    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:55.839568 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:55.839469    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:58:55.839568 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:55.839537    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:58:55.839661 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:55.839631    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:58:57.840770 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:57.840671    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:57.840770 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:57.840770    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:57.840770 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:57.840782    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:58:57.841405 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:57.840873    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:58:57.841405 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:57.840926    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:57.841405 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:57.841013    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:58:59.840180 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:59.839952    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:58:59.840763 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:59.839984    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:58:59.840763 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:59.840288    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:58:59.840763 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:58:59.840071    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:58:59.840763 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:59.840360    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:58:59.840763 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:58:59.840461    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:59:00.050936 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.050722    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 04:59:00.051285 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.051261    2562 generic.go:356] "Generic (PLEG): container finished" podID="e1aacd1d-1083-4b6d-8eb6-064cb6be4430" containerID="5f5449b76b77e7282114d8072211d753f4a26749e962234a03d49afa3f93feab" exitCode=1
Jul 15 04:59:00.051408 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.051341    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c" event={"ID":"e1aacd1d-1083-4b6d-8eb6-064cb6be4430","Type":"ContainerStarted","Data":"324220d72b81426d705379d80694fc7a502365ccda709350a6e68eb68aced86f"}
Jul 15 04:59:00.051459 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.051440    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c" event={"ID":"e1aacd1d-1083-4b6d-8eb6-064cb6be4430","Type":"ContainerStarted","Data":"af421e6d50d157cf5171947eacb4a6673db77f5ece4d5480c6500d61b737fcc3"}
Jul 15 04:59:00.051504 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.051492    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c" event={"ID":"e1aacd1d-1083-4b6d-8eb6-064cb6be4430","Type":"ContainerStarted","Data":"053b19c1995a9cec85625c47f7deb4e928e7cc39a88a69163f437d1ee2b88e6b"}
Jul 15 04:59:00.051561 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.051536    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c" event={"ID":"e1aacd1d-1083-4b6d-8eb6-064cb6be4430","Type":"ContainerDied","Data":"5f5449b76b77e7282114d8072211d753f4a26749e962234a03d49afa3f93feab"}
Jul 15 04:59:00.051606 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.051575    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c" event={"ID":"e1aacd1d-1083-4b6d-8eb6-064cb6be4430","Type":"ContainerStarted","Data":"bbb4ec1ed2be9bb82ed8c75aad5f23e189c5ac193125f51922b0fd1e871145f6"}
Jul 15 04:59:00.053071 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.052926    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c8jpr" event={"ID":"f0bc6afb-365d-4ab0-8088-a67c2c9a4316","Type":"ContainerStarted","Data":"db6cb1aeb2099ac2de7548d39df06171d390c673c5364b6b64a201e8b85a5295"}
Jul 15 04:59:00.054622 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.054598    2562 generic.go:356] "Generic (PLEG): container finished" podID="24d3a875-2a2f-47a9-9842-c12d04ad9fb9" containerID="2f49751551786c1c63d962b88de246bed9fa0b30dfba40439011c218ff1da5bd" exitCode=0
Jul 15 04:59:00.054727 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.054680    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b46gk" event={"ID":"24d3a875-2a2f-47a9-9842-c12d04ad9fb9","Type":"ContainerDied","Data":"2f49751551786c1c63d962b88de246bed9fa0b30dfba40439011c218ff1da5bd"}
Jul 15 04:59:00.056762 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.056731    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gp2nd" event={"ID":"e49b01bc-0081-4961-90a6-0b2fd594abf7","Type":"ContainerStarted","Data":"c214d36cda3accc6ede00de6d400246b6177cf6a6b1ed77cc11c3383d98e1ce8"}
Jul 15 04:59:00.058232 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.058207    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4tjhh" event={"ID":"b13f25c6-5faa-4d10-94e9-1174def82690","Type":"ContainerStarted","Data":"59b0f7b7e7297667a4eee1c69eb97eafc67987cc733565f400539cb8aba0d1d4"}
Jul 15 04:59:00.059605 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.059578    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6tc82" event={"ID":"e3d94ce3-5595-45ad-90e8-b5da6d480cc5","Type":"ContainerStarted","Data":"9cf12f4a4daf7bc46254045509ad60c801cd3c2a48f6384a44e19f972785c029"}
Jul 15 04:59:00.061112 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.061076    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5" event={"ID":"3d04b791-6db8-4b54-9670-cc198c1868af","Type":"ContainerStarted","Data":"671c034890959eaee00ec57b8561a06351df7c4172d8da20c06852f2c6cfc7db"}
Jul 15 04:59:00.062326 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.062309    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6rksr" event={"ID":"674d12b8-5912-4f32-b965-1a498763f81c","Type":"ContainerStarted","Data":"c85dcac788ce2cd0cbb4fa2d7727b2d24756274deb05add9fa89484f325ddb9f"}
Jul 15 04:59:00.071516 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.071482    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-231.ec2.internal" podStartSLOduration=21.071470564 podStartE2EDuration="21.071470564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:58:43.030698458 +0000 UTC m=+5.634251066" watchObservedRunningTime="2026-07-15 04:59:00.071470564 +0000 UTC m=+22.675023172"
Jul 15 04:59:00.072061 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.072033    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c8jpr" podStartSLOduration=3.708123279 podStartE2EDuration="22.072024299s" totalImagesPullingTime="18.36390102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:00.071593199 +0000 UTC m=+22.675145794" watchObservedRunningTime="2026-07-15 04:59:00.072024299 +0000 UTC m=+22.675576906"
Jul 15 04:59:00.110382 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.110325    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4tjhh" podStartSLOduration=3.770176197 podStartE2EDuration="22.11030604s" totalImagesPullingTime="18.340129843s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:00.0926999 +0000 UTC m=+22.696252509" watchObservedRunningTime="2026-07-15 04:59:00.11030604 +0000 UTC m=+22.713858651"
Jul 15 04:59:00.110502 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.110410    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6tc82" podStartSLOduration=3.795642317 podStartE2EDuration="22.110402963s" totalImagesPullingTime="18.314760646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:00.109706259 +0000 UTC m=+22.713258877" watchObservedRunningTime="2026-07-15 04:59:00.110402963 +0000 UTC m=+22.713955572"
Jul 15 04:59:00.149565 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.149448    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gp2nd" podStartSLOduration=3.832597398 podStartE2EDuration="22.149428851s" totalImagesPullingTime="18.316831453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:00.149179396 +0000 UTC m=+22.752732003" watchObservedRunningTime="2026-07-15 04:59:00.149428851 +0000 UTC m=+22.752981463"
Jul 15 04:59:00.165328 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.165282    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6rksr" podStartSLOduration=3.65437768 podStartE2EDuration="22.165266864s" totalImagesPullingTime="18.510889184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:00.164913462 +0000 UTC m=+22.768466069" watchObservedRunningTime="2026-07-15 04:59:00.165266864 +0000 UTC m=+22.768819472"
Jul 15 04:59:00.890247 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:00.890206    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 04:59:01.067498 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:01.067420    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5" event={"ID":"3d04b791-6db8-4b54-9670-cc198c1868af","Type":"ContainerStarted","Data":"8a20eb561879f593be7114e49370f83e1228ba5f331aa088893e37f58b398c66"}
Jul 15 04:59:01.068956 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:01.068923    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vq49q" event={"ID":"9af2c199-0437-4e1a-ae12-0113b8e325b9","Type":"ContainerStarted","Data":"0f68f6ae5d7b42ab533bf5d4d7d442cfeb2618e1861fcd28ed9f2cb5c9dac51e"}
Jul 15 04:59:01.072271 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:01.072248    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 04:59:01.072817 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:01.072790    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c" event={"ID":"e1aacd1d-1083-4b6d-8eb6-064cb6be4430","Type":"ContainerStarted","Data":"f8da7129e27f26d4e6f15278b2f36a32164b0f5709b7f7ca6fd455145371ce60"}
Jul 15 04:59:01.086272 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:01.086192    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vq49q" podStartSLOduration=4.711638603 podStartE2EDuration="23.086171433s" totalImagesPullingTime="18.37453283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:01.085320295 +0000 UTC m=+23.688872913" watchObservedRunningTime="2026-07-15 04:59:01.086171433 +0000 UTC m=+23.689724035"
Jul 15 04:59:01.745132 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:01.745014    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T04:59:00.890228174Z","UUID":"5bf35c6b-9e56-4604-82a5-b2ab8972fc04","Handler":null,"Name":"","Endpoint":""}
Jul 15 04:59:01.746676 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:01.746651    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 04:59:01.746803 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:01.746698    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 04:59:01.839793 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:01.839765    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:59:01.839974 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:01.839772    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:59:01.839974 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:01.839900    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:59:01.840104 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:01.839984    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:59:01.840104 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:01.840046    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:59:01.840188 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:01.840121    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:59:02.048501 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:02.048428    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gp2nd"
Jul 15 04:59:02.049142 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:02.049120    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gp2nd"
Jul 15 04:59:02.077072 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:02.077036    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5" event={"ID":"3d04b791-6db8-4b54-9670-cc198c1868af","Type":"ContainerStarted","Data":"733d3d2964a37fc93052624d5bbcb57c0b30b225cf523a7bbc0899f301cae0f2"}
Jul 15 04:59:02.101304 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:02.101255    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fd7t5" podStartSLOduration=3.657031431 podStartE2EDuration="24.101240561s" totalImagesPullingTime="20.44420913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:02.100932133 +0000 UTC m=+24.704484740" watchObservedRunningTime="2026-07-15 04:59:02.101240561 +0000 UTC m=+24.704793225"
Jul 15 04:59:03.083569 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:03.083396    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 04:59:03.084051 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:03.084014    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c" event={"ID":"e1aacd1d-1083-4b6d-8eb6-064cb6be4430","Type":"ContainerStarted","Data":"70bc80da06f5e505a16436a2ce4d21df0263636e1ca89de3bc702e03a9e6cf23"}
Jul 15 04:59:03.084161 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:03.084078    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 15 04:59:03.839776 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:03.839742    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:59:03.839956 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:03.839742    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:59:03.839956 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:03.839870    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:59:03.839956 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:03.839744    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:59:03.839956 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:03.839950    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:59:03.840126 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:03.840022    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:59:05.091505 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:05.091321    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 04:59:05.092310 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:05.091829    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c" event={"ID":"e1aacd1d-1083-4b6d-8eb6-064cb6be4430","Type":"ContainerStarted","Data":"85cfd44fc7a2967bc3d801eea2ad64cd1f7dc9cd2bf6c16a1ead9d1fa610cffa"}
Jul 15 04:59:05.092310 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:05.092087    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:59:05.092452 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:05.092381    2562 scope.go:117] "RemoveContainer" containerID="5f5449b76b77e7282114d8072211d753f4a26749e962234a03d49afa3f93feab"
Jul 15 04:59:05.093698 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:05.093672    2562 generic.go:356] "Generic (PLEG): container finished" podID="24d3a875-2a2f-47a9-9842-c12d04ad9fb9" containerID="edd5429042920d3e89de5c82d77f3b721ee123c1f01155d98635bfdeb264ae23" exitCode=0
Jul 15 04:59:05.093816 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:05.093706    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b46gk" event={"ID":"24d3a875-2a2f-47a9-9842-c12d04ad9fb9","Type":"ContainerDied","Data":"edd5429042920d3e89de5c82d77f3b721ee123c1f01155d98635bfdeb264ae23"}
Jul 15 04:59:05.108266 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:05.108244    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:59:05.839433 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:05.839393    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:59:05.839605 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:05.839438    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:59:05.839605 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:05.839403    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:59:05.839605 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:05.839505    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:59:05.839745 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:05.839602    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:59:05.839745 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:05.839670    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:59:06.099494 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.099467    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 04:59:06.099959 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.099831    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c" event={"ID":"e1aacd1d-1083-4b6d-8eb6-064cb6be4430","Type":"ContainerStarted","Data":"f89e1724b8a475c1f3811b9295e4f4d3723ccdde394c32990885243a58b24e3b"}
Jul 15 04:59:06.100203 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.100178    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:59:06.100334 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.100245    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:59:06.115281 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.115258    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:59:06.129645 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.129602    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c" podStartSLOduration=9.729900613 podStartE2EDuration="28.1295879s" totalImagesPullingTime="18.399687287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:06.12789683 +0000 UTC m=+28.731449448" watchObservedRunningTime="2026-07-15 04:59:06.1295879 +0000 UTC m=+28.733140508"
Jul 15 04:59:06.410140 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.410113    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gp2nd"
Jul 15 04:59:06.410301 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.410287    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 15 04:59:06.410672 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.410650    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gp2nd"
Jul 15 04:59:06.705463 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.705220    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tkz8s"]
Jul 15 04:59:06.705628 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.705556    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:59:06.705671 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:06.705644    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:59:06.705760 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.705744    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v8c4q"]
Jul 15 04:59:06.705844 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.705834    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:59:06.705922 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:06.705904    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:59:06.707475 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.707450    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9vxv5"]
Jul 15 04:59:06.707634 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:06.707543    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:59:06.707704 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:06.707631    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:59:07.103515 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:07.103423    2562 generic.go:356] "Generic (PLEG): container finished" podID="24d3a875-2a2f-47a9-9842-c12d04ad9fb9" containerID="233114d143111c88572118756d21f0e09e6be73865964a366079f74110c45b26" exitCode=0
Jul 15 04:59:07.104063 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:07.103517    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b46gk" event={"ID":"24d3a875-2a2f-47a9-9842-c12d04ad9fb9","Type":"ContainerDied","Data":"233114d143111c88572118756d21f0e09e6be73865964a366079f74110c45b26"}
Jul 15 04:59:07.840222 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:07.840191    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:59:07.840403 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:07.840289    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:59:08.839809 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:08.839724    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:59:08.840337 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:08.839724    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:59:08.840337 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:08.839842    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:59:08.840337 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:08.839896    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:59:09.111285 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:09.111255    2562 generic.go:356] "Generic (PLEG): container finished" podID="24d3a875-2a2f-47a9-9842-c12d04ad9fb9" containerID="cada87eba39a195de11d31138e4ea4298539ec9392fd503305c4888261b1d629" exitCode=0
Jul 15 04:59:09.111427 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:09.111324    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b46gk" event={"ID":"24d3a875-2a2f-47a9-9842-c12d04ad9fb9","Type":"ContainerDied","Data":"cada87eba39a195de11d31138e4ea4298539ec9392fd503305c4888261b1d629"}
Jul 15 04:59:09.839939 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:09.839904    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:59:09.840537 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:09.840035    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:59:10.839824 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:10.839783    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:59:10.839988 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:10.839791    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:59:10.839988 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:10.839930    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:59:10.839988 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:10.839981    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:59:11.542282 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:11.542245    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8m9g\" (UniqueName: \"kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g\") pod \"network-check-target-v8c4q\" (UID: \"8bd2c88a-1300-41b1-984f-fc4d17ff45f6\") " pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:59:11.542487 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:11.542294    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:59:11.542487 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:11.542317    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:59:11.542487 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:11.542445    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:59:11.542487 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:11.542467    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 04:59:11.542650 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:11.542458    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 04:59:11.542650 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:11.542515    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs podName:30b62c6b-b454-403b-b1be-4fd921f4a5e4 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:43.542498141 +0000 UTC m=+66.146050727 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs") pod "network-metrics-daemon-9vxv5" (UID: "30b62c6b-b454-403b-b1be-4fd921f4a5e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 04:59:11.542650 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:11.542525    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 04:59:11.542650 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:11.542534    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret podName:7dced01d-9de8-4e21-8ca3-241d08ecab2e nodeName:}" failed. No retries permitted until 2026-07-15 04:59:43.542525244 +0000 UTC m=+66.146077831 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret") pod "global-pull-secret-syncer-tkz8s" (UID: "7dced01d-9de8-4e21-8ca3-241d08ecab2e") : object "kube-system"/"original-pull-secret" not registered
Jul 15 04:59:11.542650 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:11.542539    2562 projected.go:196] Error preparing data for projected volume kube-api-access-r8m9g for pod openshift-network-diagnostics/network-check-target-v8c4q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:59:11.542650 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:11.542584    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g podName:8bd2c88a-1300-41b1-984f-fc4d17ff45f6 nodeName:}" failed. No retries permitted until 2026-07-15 04:59:43.542568455 +0000 UTC m=+66.146121044 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8m9g" (UniqueName: "kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g") pod "network-check-target-v8c4q" (UID: "8bd2c88a-1300-41b1-984f-fc4d17ff45f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 04:59:11.839991 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:11.839956    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:59:11.840448 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:11.840090    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v8c4q" podUID="8bd2c88a-1300-41b1-984f-fc4d17ff45f6"
Jul 15 04:59:12.839312 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:12.839105    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:59:12.839533 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:12.839107    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:59:12.839533 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:12.839447    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 04:59:12.839533 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:12.839496    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkz8s" podUID="7dced01d-9de8-4e21-8ca3-241d08ecab2e"
Jul 15 04:59:13.286588 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.286560    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-231.ec2.internal" event="NodeReady"
Jul 15 04:59:13.287053 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.286715    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 04:59:13.331996 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.331968    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bhmbw"]
Jul 15 04:59:13.334292 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.334263    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:13.335217 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.335195    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p7b4f"]
Jul 15 04:59:13.336720 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.336695    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p7b4f"
Jul 15 04:59:13.337237 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.337214    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 04:59:13.337355 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.337220    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-crtzp\""
Jul 15 04:59:13.337613 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.337589    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 04:59:13.339834 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.339815    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-22jq7\""
Jul 15 04:59:13.339975 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.339957    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 04:59:13.340093 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.340079    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 04:59:13.340193 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.340173    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 04:59:13.354035 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.351242    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p7b4f"]
Jul 15 04:59:13.355624 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.355600    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bhmbw"]
Jul 15 04:59:13.459120 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.459083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be469964-3786-4205-a37b-5a0198eed10d-tmp-dir\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:13.459120 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.459124    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:13.459359 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.459146    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/be469964-3786-4205-a37b-5a0198eed10d-config-volume\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:13.459359 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.459235    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrqdj\" (UniqueName: \"kubernetes.io/projected/be469964-3786-4205-a37b-5a0198eed10d-kube-api-access-mrqdj\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:13.459359 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.459337    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s25jh\" (UniqueName: \"kubernetes.io/projected/c99848fc-5c0a-4482-8535-10bdd21098d2-kube-api-access-s25jh\") pod \"ingress-canary-p7b4f\" (UID: \"c99848fc-5c0a-4482-8535-10bdd21098d2\") " pod="openshift-ingress-canary/ingress-canary-p7b4f"
Jul 15 04:59:13.459522 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.459430    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c99848fc-5c0a-4482-8535-10bdd21098d2-cert\") pod \"ingress-canary-p7b4f\" (UID: \"c99848fc-5c0a-4482-8535-10bdd21098d2\") " pod="openshift-ingress-canary/ingress-canary-p7b4f"
Jul 15 04:59:13.560255 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.560163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c99848fc-5c0a-4482-8535-10bdd21098d2-cert\") pod \"ingress-canary-p7b4f\" (UID: \"c99848fc-5c0a-4482-8535-10bdd21098d2\") " pod="openshift-ingress-canary/ingress-canary-p7b4f"
Jul 15 04:59:13.560255 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.560202    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be469964-3786-4205-a37b-5a0198eed10d-tmp-dir\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:13.560255 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.560219    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:13.560552 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.560346    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/be469964-3786-4205-a37b-5a0198eed10d-config-volume\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:13.560552 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.560419    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrqdj\" (UniqueName: \"kubernetes.io/projected/be469964-3786-4205-a37b-5a0198eed10d-kube-api-access-mrqdj\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:13.560552 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:13.560461    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:13.560552 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.560506    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be469964-3786-4205-a37b-5a0198eed10d-tmp-dir\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:13.560552 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:13.560529    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls podName:be469964-3786-4205-a37b-5a0198eed10d nodeName:}" failed. No retries permitted until 2026-07-15 04:59:14.060509687 +0000 UTC m=+36.664062289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls") pod "dns-default-bhmbw" (UID: "be469964-3786-4205-a37b-5a0198eed10d") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:13.560552 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.560464    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s25jh\" (UniqueName: \"kubernetes.io/projected/c99848fc-5c0a-4482-8535-10bdd21098d2-kube-api-access-s25jh\") pod \"ingress-canary-p7b4f\" (UID: \"c99848fc-5c0a-4482-8535-10bdd21098d2\") " pod="openshift-ingress-canary/ingress-canary-p7b4f"
Jul 15 04:59:13.561032 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.561003    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/be469964-3786-4205-a37b-5a0198eed10d-config-volume\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:13.565420 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.565395    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c99848fc-5c0a-4482-8535-10bdd21098d2-cert\") pod \"ingress-canary-p7b4f\" (UID: \"c99848fc-5c0a-4482-8535-10bdd21098d2\") " pod="openshift-ingress-canary/ingress-canary-p7b4f"
Jul 15 04:59:13.569651 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.569606    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrqdj\" (UniqueName: \"kubernetes.io/projected/be469964-3786-4205-a37b-5a0198eed10d-kube-api-access-mrqdj\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:13.569861 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.569822    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s25jh\" (UniqueName: \"kubernetes.io/projected/c99848fc-5c0a-4482-8535-10bdd21098d2-kube-api-access-s25jh\") pod \"ingress-canary-p7b4f\" (UID: \"c99848fc-5c0a-4482-8535-10bdd21098d2\") " pod="openshift-ingress-canary/ingress-canary-p7b4f"
Jul 15 04:59:13.618659 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.618562    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:13.657469 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.657423    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p7b4f"
Jul 15 04:59:13.706317 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.706280    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:13.764070 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.764036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:13.839193 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.839103    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:59:13.842619 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.842407    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lb4wr\""
Jul 15 04:59:13.842619 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.842413    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 04:59:13.842619 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:13.842479    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 04:59:14.063550 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:14.063513    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:14.063717 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:14.063688    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:14.063758 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:14.063753    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls podName:be469964-3786-4205-a37b-5a0198eed10d nodeName:}" failed. No retries permitted until 2026-07-15 04:59:15.063736387 +0000 UTC m=+37.667288973 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls") pod "dns-default-bhmbw" (UID: "be469964-3786-4205-a37b-5a0198eed10d") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:14.154228 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:14.154200    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p7b4f"]
Jul 15 04:59:14.157148 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:59:14.157120    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc99848fc_5c0a_4482_8535_10bdd21098d2.slice/crio-fdb192fc6c688cfbfe1934632a6be9cbb8cd96336d38a82c44906f33a88736a0 WatchSource:0}: Error finding container fdb192fc6c688cfbfe1934632a6be9cbb8cd96336d38a82c44906f33a88736a0: Status 404 returned error can't find the container with id fdb192fc6c688cfbfe1934632a6be9cbb8cd96336d38a82c44906f33a88736a0
Jul 15 04:59:14.839856 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:14.839775    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:59:14.840286 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:14.839947    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:59:14.842887 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:14.842864    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 04:59:14.843954 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:14.843933    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 04:59:14.844066 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:14.843933    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9bt4r\""
Jul 15 04:59:15.069291 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:15.069257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:15.069478 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:15.069405    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:15.069478 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:15.069467    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls podName:be469964-3786-4205-a37b-5a0198eed10d nodeName:}" failed. No retries permitted until 2026-07-15 04:59:17.069451297 +0000 UTC m=+39.673003884 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls") pod "dns-default-bhmbw" (UID: "be469964-3786-4205-a37b-5a0198eed10d") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:15.129207 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:15.128958    2562 generic.go:356] "Generic (PLEG): container finished" podID="24d3a875-2a2f-47a9-9842-c12d04ad9fb9" containerID="a2f2d742cd7e754209737302072f76f4f1cabfecf00cd6ba020c2e0790097920" exitCode=0
Jul 15 04:59:15.129207 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:15.129038    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b46gk" event={"ID":"24d3a875-2a2f-47a9-9842-c12d04ad9fb9","Type":"ContainerDied","Data":"a2f2d742cd7e754209737302072f76f4f1cabfecf00cd6ba020c2e0790097920"}
Jul 15 04:59:15.130743 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:15.130716    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p7b4f" event={"ID":"c99848fc-5c0a-4482-8535-10bdd21098d2","Type":"ContainerStarted","Data":"fdb192fc6c688cfbfe1934632a6be9cbb8cd96336d38a82c44906f33a88736a0"}
Jul 15 04:59:15.260497 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:15.260467    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:15.330596 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:15.330560    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:16.135824 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:16.135789    2562 generic.go:356] "Generic (PLEG): container finished" podID="24d3a875-2a2f-47a9-9842-c12d04ad9fb9" containerID="163795a916989a5aec5ec10018076f1166a0aeebc41e64d09115a3a937399fd9" exitCode=0
Jul 15 04:59:16.136308 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:16.135862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b46gk" event={"ID":"24d3a875-2a2f-47a9-9842-c12d04ad9fb9","Type":"ContainerDied","Data":"163795a916989a5aec5ec10018076f1166a0aeebc41e64d09115a3a937399fd9"}
Jul 15 04:59:17.083803 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:17.083767    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:17.083982 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:17.083874    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:17.083982 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:17.083927    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls podName:be469964-3786-4205-a37b-5a0198eed10d nodeName:}" failed. No retries permitted until 2026-07-15 04:59:21.083912525 +0000 UTC m=+43.687465112 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls") pod "dns-default-bhmbw" (UID: "be469964-3786-4205-a37b-5a0198eed10d") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:17.141749 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:17.141711    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b46gk" event={"ID":"24d3a875-2a2f-47a9-9842-c12d04ad9fb9","Type":"ContainerStarted","Data":"462491b85fb40ce34731a089bab49551502a0541f3b80c083dc5b2e8e4f30711"}
Jul 15 04:59:17.143173 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:17.143143    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p7b4f" event={"ID":"c99848fc-5c0a-4482-8535-10bdd21098d2","Type":"ContainerStarted","Data":"5b71c2994ec75fbd18e12eafd47c820e2ff8732bdbb08e71df48e0be127df2a6"}
Jul 15 04:59:17.166745 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:17.166695    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b46gk" podStartSLOduration=9.244191079 podStartE2EDuration="39.166678401s" totalImagesPullingTime="29.922487322s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:17.165109303 +0000 UTC m=+39.768661912" watchObservedRunningTime="2026-07-15 04:59:17.166678401 +0000 UTC m=+39.770231008"
Jul 15 04:59:17.180142 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:17.180092    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p7b4f" podStartSLOduration=1.918254686 podStartE2EDuration="4.180075735s" totalImagesPullingTime="2.261821049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:17.179181494 +0000 UTC m=+39.782734092" watchObservedRunningTime="2026-07-15 04:59:17.180075735 +0000 UTC m=+39.783628321"
Jul 15 04:59:17.294835 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:17.294803    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:17.359783 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:17.359704    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:17.631682 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:17.631603    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:17.687467 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:17.687426    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:20.765214 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:20.765176    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:20.865722 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:20.865683    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:21.113788 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:21.113700    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:21.113951 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:21.113811    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:21.113951 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:21.113863    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls podName:be469964-3786-4205-a37b-5a0198eed10d nodeName:}" failed. No retries permitted until 2026-07-15 04:59:29.113849823 +0000 UTC m=+51.717402409 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls") pod "dns-default-bhmbw" (UID: "be469964-3786-4205-a37b-5a0198eed10d") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:22.337707 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:22.337671    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:29.174537 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:29.174503    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:29.175022 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:29.174639    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:29.175022 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:29.174703    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls podName:be469964-3786-4205-a37b-5a0198eed10d nodeName:}" failed. No retries permitted until 2026-07-15 04:59:45.174688374 +0000 UTC m=+67.778240961 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls") pod "dns-default-bhmbw" (UID: "be469964-3786-4205-a37b-5a0198eed10d") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:37.236285 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:37.236209    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:37.300054 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:37.300021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:38.116184 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:38.116154    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djb4c"
Jul 15 04:59:38.266378 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:38.266342    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:38.321654 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:38.321619    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:42.709180 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:42.709130    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:42.769593 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:42.769560    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:42.820281 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:42.820234    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:43.572082 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.572029    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8m9g\" (UniqueName: \"kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g\") pod \"network-check-target-v8c4q\" (UID: \"8bd2c88a-1300-41b1-984f-fc4d17ff45f6\") " pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:59:43.572082 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.572091    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 04:59:43.572365 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.572118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:59:43.574845 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.574826    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 04:59:43.574845 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.574833    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 04:59:43.574922 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.574867    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 04:59:43.582983 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:43.582962    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 15 04:59:43.583089 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:43.583040    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs podName:30b62c6b-b454-403b-b1be-4fd921f4a5e4 nodeName:}" failed. No retries permitted until 2026-07-15 05:00:47.583018744 +0000 UTC m=+130.186571331 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs") pod "network-metrics-daemon-9vxv5" (UID: "30b62c6b-b454-403b-b1be-4fd921f4a5e4") : secret "metrics-daemon-secret" not found
Jul 15 04:59:43.585190 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.585172    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 04:59:43.586190 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.586169    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7dced01d-9de8-4e21-8ca3-241d08ecab2e-original-pull-secret\") pod \"global-pull-secret-syncer-tkz8s\" (UID: \"7dced01d-9de8-4e21-8ca3-241d08ecab2e\") " pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:59:43.595078 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.595058    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8m9g\" (UniqueName: \"kubernetes.io/projected/8bd2c88a-1300-41b1-984f-fc4d17ff45f6-kube-api-access-r8m9g\") pod \"network-check-target-v8c4q\" (UID: \"8bd2c88a-1300-41b1-984f-fc4d17ff45f6\") " pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:59:43.650318 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.650276    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkz8s"
Jul 15 04:59:43.793030 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.793000    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tkz8s"]
Jul 15 04:59:43.796687 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:59:43.796658    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dced01d_9de8_4e21_8ca3_241d08ecab2e.slice/crio-879e31a9c6c7b9a719de27656cf36be0b0e4a65a0eba47f2641c7c2b6cfb5937 WatchSource:0}: Error finding container 879e31a9c6c7b9a719de27656cf36be0b0e4a65a0eba47f2641c7c2b6cfb5937: Status 404 returned error can't find the container with id 879e31a9c6c7b9a719de27656cf36be0b0e4a65a0eba47f2641c7c2b6cfb5937
Jul 15 04:59:43.852922 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.852843    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lb4wr\""
Jul 15 04:59:43.860822 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.860804    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:59:43.976907 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:43.976879    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v8c4q"]
Jul 15 04:59:43.979654 ip-10-0-128-231 kubenswrapper[2562]: W0715 04:59:43.979627    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bd2c88a_1300_41b1_984f_fc4d17ff45f6.slice/crio-a9d300912b0ed334ae8069babe907cf16e7ad51a17186a48777f6e35d285e850 WatchSource:0}: Error finding container a9d300912b0ed334ae8069babe907cf16e7ad51a17186a48777f6e35d285e850: Status 404 returned error can't find the container with id a9d300912b0ed334ae8069babe907cf16e7ad51a17186a48777f6e35d285e850
Jul 15 04:59:44.207112 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:44.207071    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v8c4q" event={"ID":"8bd2c88a-1300-41b1-984f-fc4d17ff45f6","Type":"ContainerStarted","Data":"a9d300912b0ed334ae8069babe907cf16e7ad51a17186a48777f6e35d285e850"}
Jul 15 04:59:44.208305 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:44.208265    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tkz8s" event={"ID":"7dced01d-9de8-4e21-8ca3-241d08ecab2e","Type":"ContainerStarted","Data":"879e31a9c6c7b9a719de27656cf36be0b0e4a65a0eba47f2641c7c2b6cfb5937"}
Jul 15 04:59:45.184862 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:45.184821    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 04:59:45.185333 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:45.185003    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 04:59:45.185333 ip-10-0-128-231 kubenswrapper[2562]: E0715 04:59:45.185087    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls podName:be469964-3786-4205-a37b-5a0198eed10d nodeName:}" failed. No retries permitted until 2026-07-15 05:00:17.185061957 +0000 UTC m=+99.788614546 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls") pod "dns-default-bhmbw" (UID: "be469964-3786-4205-a37b-5a0198eed10d") : secret "dns-default-metrics-tls" not found
Jul 15 04:59:49.223427 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:49.223389    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v8c4q" event={"ID":"8bd2c88a-1300-41b1-984f-fc4d17ff45f6","Type":"ContainerStarted","Data":"6afd831ea5b6bc72f20b51e810a110923aa99414ae58bc773292d501a0ea94fa"}
Jul 15 04:59:49.223906 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:49.223459    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 04:59:49.224746 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:49.224720    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tkz8s" event={"ID":"7dced01d-9de8-4e21-8ca3-241d08ecab2e","Type":"ContainerStarted","Data":"289841f567cb61367bc8c96e4a5b25e7504ebc0f278d84acaf92f33b10ebd459"}
Jul 15 04:59:49.238946 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:49.238897    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-v8c4q" podStartSLOduration=66.877596778 podStartE2EDuration="1m11.238881928s" totalImagesPullingTime="4.36128515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:49.23841518 +0000 UTC m=+71.841967784" watchObservedRunningTime="2026-07-15 04:59:49.238881928 +0000 UTC m=+71.842434537"
Jul 15 04:59:49.253176 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:49.253127    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tkz8s" podStartSLOduration=66.704727631 podStartE2EDuration="1m11.253113404s" totalImagesPullingTime="4.548385773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 04:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 04:59:49.253042342 +0000 UTC m=+71.856594951" watchObservedRunningTime="2026-07-15 04:59:49.253113404 +0000 UTC m=+71.856666011"
Jul 15 04:59:50.853017 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:50.852981    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 04:59:50.906243 ip-10-0-128-231 kubenswrapper[2562]: I0715 04:59:50.906212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:00:13.532618 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:13.532497    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:00:13.937145 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:13.937072    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"]
Jul 15 05:00:13.941195 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:13.941176    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:00:13.943816 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:13.943797    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 15 05:00:13.944972 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:13.944953    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 15 05:00:13.945081 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:13.944982    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 15 05:00:13.945081 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:13.944993    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 15 05:00:13.952722 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:13.952704    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"]
Jul 15 05:00:13.978132 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:13.978103    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edb5bd33-67fb-410b-a0f5-a41c07d7f03a-tmp\") pod \"klusterlet-addon-workmgr-656fc9d9c6-nk2vf\" (UID: \"edb5bd33-67fb-410b-a0f5-a41c07d7f03a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:00:13.978262 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:13.978211    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/edb5bd33-67fb-410b-a0f5-a41c07d7f03a-klusterlet-config\") pod \"klusterlet-addon-workmgr-656fc9d9c6-nk2vf\" (UID: \"edb5bd33-67fb-410b-a0f5-a41c07d7f03a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:00:13.978262 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:13.978242    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl4pm\" (UniqueName: \"kubernetes.io/projected/edb5bd33-67fb-410b-a0f5-a41c07d7f03a-kube-api-access-pl4pm\") pod \"klusterlet-addon-workmgr-656fc9d9c6-nk2vf\" (UID: \"edb5bd33-67fb-410b-a0f5-a41c07d7f03a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:00:14.078560 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:14.078523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/edb5bd33-67fb-410b-a0f5-a41c07d7f03a-klusterlet-config\") pod \"klusterlet-addon-workmgr-656fc9d9c6-nk2vf\" (UID: \"edb5bd33-67fb-410b-a0f5-a41c07d7f03a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:00:14.078560 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:14.078560    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl4pm\" (UniqueName: \"kubernetes.io/projected/edb5bd33-67fb-410b-a0f5-a41c07d7f03a-kube-api-access-pl4pm\") pod \"klusterlet-addon-workmgr-656fc9d9c6-nk2vf\" (UID: \"edb5bd33-67fb-410b-a0f5-a41c07d7f03a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:00:14.078751 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:14.078592    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edb5bd33-67fb-410b-a0f5-a41c07d7f03a-tmp\") pod \"klusterlet-addon-workmgr-656fc9d9c6-nk2vf\" (UID: \"edb5bd33-67fb-410b-a0f5-a41c07d7f03a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:00:14.079030 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:14.079009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edb5bd33-67fb-410b-a0f5-a41c07d7f03a-tmp\") pod \"klusterlet-addon-workmgr-656fc9d9c6-nk2vf\" (UID: \"edb5bd33-67fb-410b-a0f5-a41c07d7f03a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:00:14.081015 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:14.081000    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/edb5bd33-67fb-410b-a0f5-a41c07d7f03a-klusterlet-config\") pod \"klusterlet-addon-workmgr-656fc9d9c6-nk2vf\" (UID: \"edb5bd33-67fb-410b-a0f5-a41c07d7f03a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:00:14.087417 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:14.087397    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl4pm\" (UniqueName: \"kubernetes.io/projected/edb5bd33-67fb-410b-a0f5-a41c07d7f03a-kube-api-access-pl4pm\") pod \"klusterlet-addon-workmgr-656fc9d9c6-nk2vf\" (UID: \"edb5bd33-67fb-410b-a0f5-a41c07d7f03a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:00:14.249472 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:14.249428    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:00:14.365316 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:14.365282    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"]
Jul 15 05:00:14.368028 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:00:14.367996    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb5bd33_67fb_410b_a0f5_a41c07d7f03a.slice/crio-1977c4d3c4083280c3de8a01196a82640067a5c8e0bbb594eeee41aab711c36d WatchSource:0}: Error finding container 1977c4d3c4083280c3de8a01196a82640067a5c8e0bbb594eeee41aab711c36d: Status 404 returned error can't find the container with id 1977c4d3c4083280c3de8a01196a82640067a5c8e0bbb594eeee41aab711c36d
Jul 15 05:00:15.285329 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:15.285292    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf" event={"ID":"edb5bd33-67fb-410b-a0f5-a41c07d7f03a","Type":"ContainerStarted","Data":"1977c4d3c4083280c3de8a01196a82640067a5c8e0bbb594eeee41aab711c36d"}
Jul 15 05:00:17.204009 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:17.203970    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 05:00:17.204450 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:00:17.204119    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 05:00:17.204450 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:00:17.204181    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls podName:be469964-3786-4205-a37b-5a0198eed10d nodeName:}" failed. No retries permitted until 2026-07-15 05:01:21.204164825 +0000 UTC m=+163.807717412 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls") pod "dns-default-bhmbw" (UID: "be469964-3786-4205-a37b-5a0198eed10d") : secret "dns-default-metrics-tls" not found
Jul 15 05:00:19.297626 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:19.297583    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf" event={"ID":"edb5bd33-67fb-410b-a0f5-a41c07d7f03a","Type":"ContainerStarted","Data":"7e214fe3f73ae5d8215a7e2915b676ec958e7f87b7a479091dfc8b21851433aa"}
Jul 15 05:00:19.298008 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:19.297800    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:00:19.299482 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:19.299461    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:00:19.315293 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:19.315247    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf" podStartSLOduration=2.210365317 podStartE2EDuration="6.315232964s" totalImagesPullingTime="4.104867647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:00:19.314806867 +0000 UTC m=+101.918359474" watchObservedRunningTime="2026-07-15 05:00:19.315232964 +0000 UTC m=+101.918785653"
Jul 15 05:00:20.229601 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:20.229567    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-v8c4q"
Jul 15 05:00:47.620964 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:00:47.620909    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 05:00:47.621430 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:00:47.621033    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 15 05:00:47.621430 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:00:47.621090    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs podName:30b62c6b-b454-403b-b1be-4fd921f4a5e4 nodeName:}" failed. No retries permitted until 2026-07-15 05:02:49.621074955 +0000 UTC m=+252.224627544 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs") pod "network-metrics-daemon-9vxv5" (UID: "30b62c6b-b454-403b-b1be-4fd921f4a5e4") : secret "metrics-daemon-secret" not found
Jul 15 05:01:16.347197 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:01:16.347144    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bhmbw" podUID="be469964-3786-4205-a37b-5a0198eed10d"
Jul 15 05:01:16.435395 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:01:16.435304    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bhmbw"
Jul 15 05:01:17.856308 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:01:17.856267    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 05:01:21.247920 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:01:21.247869    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 05:01:21.248329 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:01:21.248018    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 05:01:21.248329 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:01:21.248090    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls podName:be469964-3786-4205-a37b-5a0198eed10d nodeName:}" failed. No retries permitted until 2026-07-15 05:03:23.248075659 +0000 UTC m=+285.851628246 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls") pod "dns-default-bhmbw" (UID: "be469964-3786-4205-a37b-5a0198eed10d") : secret "dns-default-metrics-tls" not found
Jul 15 05:01:30.839310 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:01:30.839217    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 05:02:18.579437 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:02:18.579402    2562 generic.go:356] "Generic (PLEG): container finished" podID="edb5bd33-67fb-410b-a0f5-a41c07d7f03a" containerID="7e214fe3f73ae5d8215a7e2915b676ec958e7f87b7a479091dfc8b21851433aa" exitCode=1
Jul 15 05:02:18.579813 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:02:18.579469    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf" event={"ID":"edb5bd33-67fb-410b-a0f5-a41c07d7f03a","Type":"ContainerDied","Data":"7e214fe3f73ae5d8215a7e2915b676ec958e7f87b7a479091dfc8b21851433aa"}
Jul 15 05:02:18.579813 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:02:18.579786    2562 scope.go:117] "RemoveContainer" containerID="7e214fe3f73ae5d8215a7e2915b676ec958e7f87b7a479091dfc8b21851433aa"
Jul 15 05:02:19.298099 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:02:19.298060    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:02:19.584498 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:02:19.584415    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf" event={"ID":"edb5bd33-67fb-410b-a0f5-a41c07d7f03a","Type":"ContainerStarted","Data":"538a72df023100db6f94643af923e9fd134b5133311d1836bb8244f1587fa378"}
Jul 15 05:02:19.584899 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:02:19.584693    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:02:19.585273 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:02:19.585259    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-656fc9d9c6-nk2vf"
Jul 15 05:02:49.718843 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:02:49.718787    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 05:02:49.719327 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:02:49.718936    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 15 05:02:49.719327 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:02:49.719003    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs podName:30b62c6b-b454-403b-b1be-4fd921f4a5e4 nodeName:}" failed. No retries permitted until 2026-07-15 05:04:51.718985643 +0000 UTC m=+374.322538234 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs") pod "network-metrics-daemon-9vxv5" (UID: "30b62c6b-b454-403b-b1be-4fd921f4a5e4") : secret "metrics-daemon-secret" not found
Jul 15 05:03:02.573724 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.573641    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"]
Jul 15 05:03:02.576479 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.576452    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c78dbcc8c-gjttn"]
Jul 15 05:03:02.576596 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.576504    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:02.579191 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.579169    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.582437 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.582251    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-fr5k9\""
Jul 15 05:03:02.582437 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.582292    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 05:03:02.582437 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.582309    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 15 05:03:02.582437 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.582328    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 15 05:03:02.582437 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.582292    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 15 05:03:02.582437 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.582398    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 15 05:03:02.582732 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.582490    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 05:03:02.582802 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.582791    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 15 05:03:02.582852 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.582835    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8tsq2\""
Jul 15 05:03:02.586705 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.586681    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 15 05:03:02.591473 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.591455    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"]
Jul 15 05:03:02.602731 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.602705    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c78dbcc8c-gjttn"]
Jul 15 05:03:02.677382 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.677339    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-zkpg2"]
Jul 15 05:03:02.680213 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.680197    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.683952 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.683928    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 15 05:03:02.684266 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.684218    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 15 05:03:02.684335 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.684254    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-c2pjr\""
Jul 15 05:03:02.684421 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.684256    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 05:03:02.684653 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.684637    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 05:03:02.689898 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.689881    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 15 05:03:02.694126 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.694105    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-zkpg2"]
Jul 15 05:03:02.706955 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.706925    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw49j\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-kube-api-access-lw49j\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.707069 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.706961    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-ca-trust-extracted\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.707069 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.706988    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6169dacf-0931-4554-b48e-5da0d8b36ddd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:02.707069 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.707059    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-certificates\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.707199 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.707085    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-trusted-ca\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.707199 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.707108    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xvt2\" (UniqueName: \"kubernetes.io/projected/6169dacf-0931-4554-b48e-5da0d8b36ddd-kube-api-access-2xvt2\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:02.707199 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.707132    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.707199 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.707147    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-installation-pull-secrets\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.707199 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.707164    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-image-registry-private-configuration\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.707421 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.707208    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-bound-sa-token\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.707421 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.707276    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:02.808180 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808123    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:02.808180 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808181    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2f8ae05-0862-4e90-bde9-1f743802015a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.808428 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808200    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chs66\" (UniqueName: \"kubernetes.io/projected/e2f8ae05-0862-4e90-bde9-1f743802015a-kube-api-access-chs66\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.808428 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:02.808292    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 15 05:03:02.808428 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808392    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw49j\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-kube-api-access-lw49j\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.808428 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:02.808406    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls podName:6169dacf-0931-4554-b48e-5da0d8b36ddd nodeName:}" failed. No retries permitted until 2026-07-15 05:03:03.308366788 +0000 UTC m=+265.911919387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-xwctc" (UID: "6169dacf-0931-4554-b48e-5da0d8b36ddd") : secret "cluster-monitoring-operator-tls" not found
Jul 15 05:03:02.808591 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808472    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-ca-trust-extracted\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.808591 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808505    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6169dacf-0931-4554-b48e-5da0d8b36ddd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:02.808591 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808532    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e2f8ae05-0862-4e90-bde9-1f743802015a-snapshots\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.808795 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808638    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2f8ae05-0862-4e90-bde9-1f743802015a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.808795 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2f8ae05-0862-4e90-bde9-1f743802015a-tmp\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.808795 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-certificates\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.808795 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808726    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-trusted-ca\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.808795 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808753    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xvt2\" (UniqueName: \"kubernetes.io/projected/6169dacf-0931-4554-b48e-5da0d8b36ddd-kube-api-access-2xvt2\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:02.808795 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.809086 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808807    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-installation-pull-secrets\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.809086 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808828    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-image-registry-private-configuration\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.809086 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808859    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-bound-sa-token\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.809086 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:02.808897    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 05:03:02.809086 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:02.808912    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c78dbcc8c-gjttn: secret "image-registry-tls" not found
Jul 15 05:03:02.809086 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.808923    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-ca-trust-extracted\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.809086 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:02.808964    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls podName:83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a nodeName:}" failed. No retries permitted until 2026-07-15 05:03:03.308946744 +0000 UTC m=+265.912499333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls") pod "image-registry-c78dbcc8c-gjttn" (UID: "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a") : secret "image-registry-tls" not found
Jul 15 05:03:02.809086 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.809056    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e2f8ae05-0862-4e90-bde9-1f743802015a-serving-cert\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.809521 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.809365    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6169dacf-0931-4554-b48e-5da0d8b36ddd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:02.809521 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.809398    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-certificates\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.809717 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.809700    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-trusted-ca\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.811222 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.811194    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-installation-pull-secrets\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.811340 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.811225    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-image-registry-private-configuration\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.817105 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.817081    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw49j\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-kube-api-access-lw49j\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.817240 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.817087    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xvt2\" (UniqueName: \"kubernetes.io/projected/6169dacf-0931-4554-b48e-5da0d8b36ddd-kube-api-access-2xvt2\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:02.817240 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.817156    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-bound-sa-token\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:02.851872 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.851790    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:02.909425 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.909389    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e2f8ae05-0862-4e90-bde9-1f743802015a-serving-cert\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.909589 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.909438    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2f8ae05-0862-4e90-bde9-1f743802015a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.909589 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.909454    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chs66\" (UniqueName: \"kubernetes.io/projected/e2f8ae05-0862-4e90-bde9-1f743802015a-kube-api-access-chs66\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.909589 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.909488    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e2f8ae05-0862-4e90-bde9-1f743802015a-snapshots\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.909589 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.909502    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2f8ae05-0862-4e90-bde9-1f743802015a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.909589 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.909521    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2f8ae05-0862-4e90-bde9-1f743802015a-tmp\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.910053 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.910031    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2f8ae05-0862-4e90-bde9-1f743802015a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.910129 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.910091    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2f8ae05-0862-4e90-bde9-1f743802015a-tmp\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.910163 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.910149    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e2f8ae05-0862-4e90-bde9-1f743802015a-snapshots\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.910348 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.910327    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2f8ae05-0862-4e90-bde9-1f743802015a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.911815 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.911797    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e2f8ae05-0862-4e90-bde9-1f743802015a-serving-cert\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.918090 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.918060    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chs66\" (UniqueName: \"kubernetes.io/projected/e2f8ae05-0862-4e90-bde9-1f743802015a-kube-api-access-chs66\") pod \"insights-operator-68d9dd94cf-zkpg2\" (UID: \"e2f8ae05-0862-4e90-bde9-1f743802015a\") " pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:02.937539 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.937506    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:02.988726 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:02.988692    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2"
Jul 15 05:03:03.109909 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:03.109835    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-zkpg2"]
Jul 15 05:03:03.112707 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:03:03.112682    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2f8ae05_0862_4e90_bde9_1f743802015a.slice/crio-ca84a72ea67982fc2237ebff47dc5fc89c848758eaa5d05de5dfc05f99e56a32 WatchSource:0}: Error finding container ca84a72ea67982fc2237ebff47dc5fc89c848758eaa5d05de5dfc05f99e56a32: Status 404 returned error can't find the container with id ca84a72ea67982fc2237ebff47dc5fc89c848758eaa5d05de5dfc05f99e56a32
Jul 15 05:03:03.312799 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:03.312756    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:03.312985 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:03.312828    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:03.312985 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:03.312916    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 15 05:03:03.313052 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:03.312985    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls podName:6169dacf-0931-4554-b48e-5da0d8b36ddd nodeName:}" failed. No retries permitted until 2026-07-15 05:03:04.312968143 +0000 UTC m=+266.916520730 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-xwctc" (UID: "6169dacf-0931-4554-b48e-5da0d8b36ddd") : secret "cluster-monitoring-operator-tls" not found
Jul 15 05:03:03.313052 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:03.312922    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 05:03:03.313052 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:03.313018    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c78dbcc8c-gjttn: secret "image-registry-tls" not found
Jul 15 05:03:03.313150 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:03.313070    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls podName:83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a nodeName:}" failed. No retries permitted until 2026-07-15 05:03:04.313055504 +0000 UTC m=+266.916608111 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls") pod "image-registry-c78dbcc8c-gjttn" (UID: "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a") : secret "image-registry-tls" not found
Jul 15 05:03:03.705785 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:03.705752    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2" event={"ID":"e2f8ae05-0862-4e90-bde9-1f743802015a","Type":"ContainerStarted","Data":"ca84a72ea67982fc2237ebff47dc5fc89c848758eaa5d05de5dfc05f99e56a32"}
Jul 15 05:03:04.319936 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:04.319907    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:04.320100 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:04.319958    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:04.320100 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:04.320053    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 15 05:03:04.320100 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:04.320077    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 05:03:04.320100 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:04.320094    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c78dbcc8c-gjttn: secret "image-registry-tls" not found
Jul 15 05:03:04.320222 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:04.320111    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls podName:6169dacf-0931-4554-b48e-5da0d8b36ddd nodeName:}" failed. No retries permitted until 2026-07-15 05:03:06.320096415 +0000 UTC m=+268.923649004 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-xwctc" (UID: "6169dacf-0931-4554-b48e-5da0d8b36ddd") : secret "cluster-monitoring-operator-tls" not found
Jul 15 05:03:04.320222 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:04.320135    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls podName:83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a nodeName:}" failed. No retries permitted until 2026-07-15 05:03:06.320120855 +0000 UTC m=+268.923673442 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls") pod "image-registry-c78dbcc8c-gjttn" (UID: "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a") : secret "image-registry-tls" not found
Jul 15 05:03:06.337688 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:06.337631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:06.338097 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:06.337713    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:06.338097 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:06.337803    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 05:03:06.338097 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:06.337823    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c78dbcc8c-gjttn: secret "image-registry-tls" not found
Jul 15 05:03:06.338097 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:06.337825    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 15 05:03:06.338097 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:06.337879    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls podName:83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a nodeName:}" failed. No retries permitted until 2026-07-15 05:03:10.337862543 +0000 UTC m=+272.941415132 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls") pod "image-registry-c78dbcc8c-gjttn" (UID: "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a") : secret "image-registry-tls" not found
Jul 15 05:03:06.338097 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:06.337892    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls podName:6169dacf-0931-4554-b48e-5da0d8b36ddd nodeName:}" failed. No retries permitted until 2026-07-15 05:03:10.337886356 +0000 UTC m=+272.941438942 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-xwctc" (UID: "6169dacf-0931-4554-b48e-5da0d8b36ddd") : secret "cluster-monitoring-operator-tls" not found
Jul 15 05:03:06.779823 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:06.779791    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:06.879315 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:06.879281    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:07.325241 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.325189    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:07.379834 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.379744    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:07.718622 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.718590    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2" event={"ID":"e2f8ae05-0862-4e90-bde9-1f743802015a","Type":"ContainerStarted","Data":"dd77d0369abac1dfbd90a970996b70a5753352a7a27c72ffcbeefea19864c8a4"}
Jul 15 05:03:07.737285 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.737238    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2" podStartSLOduration=1.759088433 podStartE2EDuration="5.737224003s" totalImagesPullingTime="3.97813557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:07.736616076 +0000 UTC m=+270.340168685" watchObservedRunningTime="2026-07-15 05:03:07.737224003 +0000 UTC m=+270.340776610"
Jul 15 05:03:07.754574 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.754543    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9xj54"]
Jul 15 05:03:07.757711 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.757693    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9xj54"
Jul 15 05:03:07.760103 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.760076    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 15 05:03:07.760237 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.760197    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 15 05:03:07.760401 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.760385    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-cfwq2\""
Jul 15 05:03:07.767201 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.767171    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9xj54"]
Jul 15 05:03:07.768643 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.768607    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:07.849901 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.849873    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gfrv\" (UniqueName: \"kubernetes.io/projected/249d0866-7a1b-44f2-a7b8-68788fb7b935-kube-api-access-8gfrv\") pod \"migrator-7f59b54f6c-9xj54\" (UID: \"249d0866-7a1b-44f2-a7b8-68788fb7b935\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9xj54"
Jul 15 05:03:07.865244 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.865212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:07.950727 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.950695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gfrv\" (UniqueName: \"kubernetes.io/projected/249d0866-7a1b-44f2-a7b8-68788fb7b935-kube-api-access-8gfrv\") pod \"migrator-7f59b54f6c-9xj54\" (UID: \"249d0866-7a1b-44f2-a7b8-68788fb7b935\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9xj54"
Jul 15 05:03:07.958907 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:07.958874    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gfrv\" (UniqueName: \"kubernetes.io/projected/249d0866-7a1b-44f2-a7b8-68788fb7b935-kube-api-access-8gfrv\") pod \"migrator-7f59b54f6c-9xj54\" (UID: \"249d0866-7a1b-44f2-a7b8-68788fb7b935\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9xj54"
Jul 15 05:03:08.036206 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.036127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:08.067437 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.067390    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9xj54"
Jul 15 05:03:08.097059 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.097015    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2dqv5"]
Jul 15 05:03:08.100316 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.100292    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.103191 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.103170    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 05:03:08.103340 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.103321    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7zgl9\""
Jul 15 05:03:08.104246 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.104229    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 05:03:08.115198 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.115139    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2dqv5"]
Jul 15 05:03:08.119546 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.119519    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:08.203856 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.203826    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9xj54"]
Jul 15 05:03:08.206908 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:03:08.206884    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod249d0866_7a1b_44f2_a7b8_68788fb7b935.slice/crio-06148c54ef99ead88b315b7ab99ba9689ea44ce841c489e2d97aa4f0a1b2df91 WatchSource:0}: Error finding container 06148c54ef99ead88b315b7ab99ba9689ea44ce841c489e2d97aa4f0a1b2df91: Status 404 returned error can't find the container with id 06148c54ef99ead88b315b7ab99ba9689ea44ce841c489e2d97aa4f0a1b2df91
Jul 15 05:03:08.252782 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.252744    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.252960 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.252786    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a222e79e-cdf0-4e4f-aa18-3a30cbede232-data-volume\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.252960 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.252874    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.252960 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.252928    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7hnr\" (UniqueName: \"kubernetes.io/projected/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-api-access-z7hnr\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.253071 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.252969    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a222e79e-cdf0-4e4f-aa18-3a30cbede232-crio-socket\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.353439 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.353324    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7hnr\" (UniqueName: \"kubernetes.io/projected/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-api-access-z7hnr\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.353439 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.353426    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a222e79e-cdf0-4e4f-aa18-3a30cbede232-crio-socket\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.353657 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.353487    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.353657 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.353513    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a222e79e-cdf0-4e4f-aa18-3a30cbede232-data-volume\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.353657 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.353543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a222e79e-cdf0-4e4f-aa18-3a30cbede232-crio-socket\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.353657 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.353558    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.353657 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:08.353607    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 05:03:08.353657 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:08.353629    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:08.353842 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:08.353670    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm podName:a222e79e-cdf0-4e4f-aa18-3a30cbede232 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:08.853654623 +0000 UTC m=+271.457207210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2dqv5" (UID: "a222e79e-cdf0-4e4f-aa18-3a30cbede232") : configmap "kube-rbac-proxy" not found
Jul 15 05:03:08.353842 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:08.353683    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls podName:a222e79e-cdf0-4e4f-aa18-3a30cbede232 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:08.853677227 +0000 UTC m=+271.457229813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2dqv5" (UID: "a222e79e-cdf0-4e4f-aa18-3a30cbede232") : secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:08.353842 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.353811    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a222e79e-cdf0-4e4f-aa18-3a30cbede232-data-volume\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.361578 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.361549    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7hnr\" (UniqueName: \"kubernetes.io/projected/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-api-access-z7hnr\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.384111 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.384084    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:08.722418 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.722363    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9xj54" event={"ID":"249d0866-7a1b-44f2-a7b8-68788fb7b935","Type":"ContainerStarted","Data":"06148c54ef99ead88b315b7ab99ba9689ea44ce841c489e2d97aa4f0a1b2df91"}
Jul 15 05:03:08.858480 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.858444    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.858645 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:08.858569    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 05:03:08.858685 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:08.858645    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm podName:a222e79e-cdf0-4e4f-aa18-3a30cbede232 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:09.858622914 +0000 UTC m=+272.462175500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2dqv5" (UID: "a222e79e-cdf0-4e4f-aa18-3a30cbede232") : configmap "kube-rbac-proxy" not found
Jul 15 05:03:08.858731 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:08.858686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:08.858844 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:08.858827    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:08.858888 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:08.858880    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls podName:a222e79e-cdf0-4e4f-aa18-3a30cbede232 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:09.858866096 +0000 UTC m=+272.462418695 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2dqv5" (UID: "a222e79e-cdf0-4e4f-aa18-3a30cbede232") : secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:09.627345 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.627315    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-6pxzb"]
Jul 15 05:03:09.630324 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.630305    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb"
Jul 15 05:03:09.632915 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.632889    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 15 05:03:09.633027 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.632889    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 15 05:03:09.633027 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.632898    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 15 05:03:09.633027 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.632898    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-8w5vl\""
Jul 15 05:03:09.633902 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.633886    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 15 05:03:09.637685 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.637668    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-6pxzb"]
Jul 15 05:03:09.667609 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.667585    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:09.725753 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.725654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:09.726792 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.726764    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9xj54" event={"ID":"249d0866-7a1b-44f2-a7b8-68788fb7b935","Type":"ContainerStarted","Data":"a6986dddd3694a46d71510c9f17947f99339e75063c0b1eff3be9b20bbed46d3"}
Jul 15 05:03:09.726908 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.726817    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9xj54" event={"ID":"249d0866-7a1b-44f2-a7b8-68788fb7b935","Type":"ContainerStarted","Data":"1a62aa9e7fb6f9891a2f03aa403743949c30753ec01ed01deba94e8aa168cbad"}
Jul 15 05:03:09.742015 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.741970    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9xj54" podStartSLOduration=1.478239973 podStartE2EDuration="2.741955525s" totalImagesPullingTime="1.263715552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:03:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:09.741240867 +0000 UTC m=+272.344793474" watchObservedRunningTime="2026-07-15 05:03:09.741955525 +0000 UTC m=+272.345508132"
Jul 15 05:03:09.766741 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.766706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l59mv\" (UniqueName: \"kubernetes.io/projected/d694d67b-0410-49d3-bcf4-28b7dc6b00b6-kube-api-access-l59mv\") pod \"service-ca-79f68b9f4f-6pxzb\" (UID: \"d694d67b-0410-49d3-bcf4-28b7dc6b00b6\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb"
Jul 15 05:03:09.766877 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.766832    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d694d67b-0410-49d3-bcf4-28b7dc6b00b6-signing-cabundle\") pod \"service-ca-79f68b9f4f-6pxzb\" (UID: \"d694d67b-0410-49d3-bcf4-28b7dc6b00b6\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb"
Jul 15 05:03:09.766877 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.766862    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d694d67b-0410-49d3-bcf4-28b7dc6b00b6-signing-key\") pod \"service-ca-79f68b9f4f-6pxzb\" (UID: \"d694d67b-0410-49d3-bcf4-28b7dc6b00b6\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb"
Jul 15 05:03:09.827845 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.827817    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:09.867570 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.867537    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:09.867809 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.867582    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l59mv\" (UniqueName: \"kubernetes.io/projected/d694d67b-0410-49d3-bcf4-28b7dc6b00b6-kube-api-access-l59mv\") pod \"service-ca-79f68b9f4f-6pxzb\" (UID: \"d694d67b-0410-49d3-bcf4-28b7dc6b00b6\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb"
Jul 15 05:03:09.867809 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.867619    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:09.867809 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.867670    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d694d67b-0410-49d3-bcf4-28b7dc6b00b6-signing-cabundle\") pod \"service-ca-79f68b9f4f-6pxzb\" (UID: \"d694d67b-0410-49d3-bcf4-28b7dc6b00b6\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb"
Jul 15 05:03:09.867809 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:09.867672    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 05:03:09.867809 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.867689    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d694d67b-0410-49d3-bcf4-28b7dc6b00b6-signing-key\") pod \"service-ca-79f68b9f4f-6pxzb\" (UID: \"d694d67b-0410-49d3-bcf4-28b7dc6b00b6\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb"
Jul 15 05:03:09.867809 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:09.867753    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm podName:a222e79e-cdf0-4e4f-aa18-3a30cbede232 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:11.867734076 +0000 UTC m=+274.471286663 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2dqv5" (UID: "a222e79e-cdf0-4e4f-aa18-3a30cbede232") : configmap "kube-rbac-proxy" not found
Jul 15 05:03:09.867809 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:09.867803    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:09.868111 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:09.867876    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls podName:a222e79e-cdf0-4e4f-aa18-3a30cbede232 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:11.867858808 +0000 UTC m=+274.471411412 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2dqv5" (UID: "a222e79e-cdf0-4e4f-aa18-3a30cbede232") : secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:09.868418 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.868401    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d694d67b-0410-49d3-bcf4-28b7dc6b00b6-signing-cabundle\") pod \"service-ca-79f68b9f4f-6pxzb\" (UID: \"d694d67b-0410-49d3-bcf4-28b7dc6b00b6\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb"
Jul 15 05:03:09.870089 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.870071    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d694d67b-0410-49d3-bcf4-28b7dc6b00b6-signing-key\") pod \"service-ca-79f68b9f4f-6pxzb\" (UID: \"d694d67b-0410-49d3-bcf4-28b7dc6b00b6\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb"
Jul 15 05:03:09.875900 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.875877    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l59mv\" (UniqueName: \"kubernetes.io/projected/d694d67b-0410-49d3-bcf4-28b7dc6b00b6-kube-api-access-l59mv\") pod \"service-ca-79f68b9f4f-6pxzb\" (UID: \"d694d67b-0410-49d3-bcf4-28b7dc6b00b6\") " pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb"
Jul 15 05:03:09.928593 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.928563    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:09.939481 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:09.939457    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb"
Jul 15 05:03:10.058029 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:10.057998    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-6pxzb"]
Jul 15 05:03:10.061062 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:03:10.061024    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd694d67b_0410_49d3_bcf4_28b7dc6b00b6.slice/crio-0b5a0c5da0affe889650a138dae38aca3233cdf3b5d3bd1e361af4bbdef3855b WatchSource:0}: Error finding container 0b5a0c5da0affe889650a138dae38aca3233cdf3b5d3bd1e361af4bbdef3855b: Status 404 returned error can't find the container with id 0b5a0c5da0affe889650a138dae38aca3233cdf3b5d3bd1e361af4bbdef3855b
Jul 15 05:03:10.372385 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:10.372270    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:10.372385 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:10.372322    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:10.372622 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:10.372474    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 05:03:10.372622 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:10.372494    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c78dbcc8c-gjttn: secret "image-registry-tls" not found
Jul 15 05:03:10.372622 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:10.372496    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 15 05:03:10.372622 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:10.372561    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls podName:83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a nodeName:}" failed. No retries permitted until 2026-07-15 05:03:18.372541325 +0000 UTC m=+280.976093924 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls") pod "image-registry-c78dbcc8c-gjttn" (UID: "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a") : secret "image-registry-tls" not found
Jul 15 05:03:10.372622 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:10.372579    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls podName:6169dacf-0931-4554-b48e-5da0d8b36ddd nodeName:}" failed. No retries permitted until 2026-07-15 05:03:18.372571058 +0000 UTC m=+280.976123647 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-xwctc" (UID: "6169dacf-0931-4554-b48e-5da0d8b36ddd") : secret "cluster-monitoring-operator-tls" not found
Jul 15 05:03:10.535506 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:10.535471    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:10.688721 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:10.688644    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4tjhh_b13f25c6-5faa-4d10-94e9-1174def82690/dns-node-resolver/0.log"
Jul 15 05:03:10.731422 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:10.731364    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb" event={"ID":"d694d67b-0410-49d3-bcf4-28b7dc6b00b6","Type":"ContainerStarted","Data":"0b5a0c5da0affe889650a138dae38aca3233cdf3b5d3bd1e361af4bbdef3855b"}
Jul 15 05:03:11.681209 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:11.681179    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6tc82_e3d94ce3-5595-45ad-90e8-b5da6d480cc5/node-ca/0.log"
Jul 15 05:03:11.735547 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:11.735508    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb" event={"ID":"d694d67b-0410-49d3-bcf4-28b7dc6b00b6","Type":"ContainerStarted","Data":"c2d99b1df19bc50fe4176bf7e64c05244743d6fb5fb38ae40742a9741ff2db92"}
Jul 15 05:03:11.753721 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:11.753668    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-6pxzb" podStartSLOduration=1.237915381 podStartE2EDuration="2.753652292s" totalImagesPullingTime="1.515736911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:03:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:11.753223074 +0000 UTC m=+274.356775704" watchObservedRunningTime="2026-07-15 05:03:11.753652292 +0000 UTC m=+274.357204899"
Jul 15 05:03:11.887564 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:11.887470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:11.887738 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:11.887614    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:11.887738 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:11.887644    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:11.887738 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:11.887698    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 05:03:11.887738 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:11.887717    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls podName:a222e79e-cdf0-4e4f-aa18-3a30cbede232 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:15.887696539 +0000 UTC m=+278.491249130 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2dqv5" (UID: "a222e79e-cdf0-4e4f-aa18-3a30cbede232") : secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:11.887893 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:11.887747    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm podName:a222e79e-cdf0-4e4f-aa18-3a30cbede232 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:15.887731344 +0000 UTC m=+278.491283930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2dqv5" (UID: "a222e79e-cdf0-4e4f-aa18-3a30cbede232") : configmap "kube-rbac-proxy" not found
Jul 15 05:03:12.426821 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:12.426789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:12.641655 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:12.641623    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:12.742057 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:12.742024    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:12.842909 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:12.842878    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:12.882060 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:12.882034    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p7b4f_c99848fc-5c0a-4482-8535-10bdd21098d2/serve-healthcheck-canary/0.log"
Jul 15 05:03:12.950331 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:12.950294    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:13.032469 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:13.032345    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:13.082388 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:13.082357    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-9xj54_249d0866-7a1b-44f2-a7b8-68788fb7b935/migrator/0.log"
Jul 15 05:03:13.128217 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:13.128183    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:13.281818 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:13.281793    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-9xj54_249d0866-7a1b-44f2-a7b8-68788fb7b935/graceful-termination/0.log"
Jul 15 05:03:13.895249 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:13.895221    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:14.142531 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:14.142500    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:14.525359 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:14.525315    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:14.728260 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:14.728228    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:14.924183 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:14.924103    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:15.330394 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:15.330339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:15.924068 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:15.924030    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:15.924242 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:15.924092    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:15.924242 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:15.924156    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 05:03:15.924242 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:15.924212    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:15.924242 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:15.924219    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm podName:a222e79e-cdf0-4e4f-aa18-3a30cbede232 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:23.924204898 +0000 UTC m=+286.527757485 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2dqv5" (UID: "a222e79e-cdf0-4e4f-aa18-3a30cbede232") : configmap "kube-rbac-proxy" not found
Jul 15 05:03:15.924631 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:15.924247    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls podName:a222e79e-cdf0-4e4f-aa18-3a30cbede232 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:23.924237351 +0000 UTC m=+286.527789937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2dqv5" (UID: "a222e79e-cdf0-4e4f-aa18-3a30cbede232") : secret "insights-runtime-extractor-tls" not found
Jul 15 05:03:16.126516 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:16.126484    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:16.227854 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:16.227825    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:16.328185 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:16.328152    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:16.537897 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:16.537816    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:16.725216 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:16.725182    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:16.921362 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:16.921283    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:17.126566 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:17.126533    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:18.129278 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:18.129242    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:18.445007 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:18.444906    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:18.445007 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:18.444972    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:18.445199 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:18.445077    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 15 05:03:18.445199 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:18.445131    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls podName:6169dacf-0931-4554-b48e-5da0d8b36ddd nodeName:}" failed. No retries permitted until 2026-07-15 05:03:34.445117183 +0000 UTC m=+297.048669770 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-xwctc" (UID: "6169dacf-0931-4554-b48e-5da0d8b36ddd") : secret "cluster-monitoring-operator-tls" not found
Jul 15 05:03:18.447228 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:18.447205    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls\") pod \"image-registry-c78dbcc8c-gjttn\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") " pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:18.492455 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:18.492419    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:18.526736 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:18.526692    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:18.621820 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:18.621779    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c78dbcc8c-gjttn"]
Jul 15 05:03:18.625282 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:03:18.625255    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83b3bfcb_ecfd_49ec_b48c_b4e6fc95dd2a.slice/crio-c0b288415af848ae6563a8097037e2b9fcc2ee702dc2d1f399cc9284fadcaf8b WatchSource:0}: Error finding container c0b288415af848ae6563a8097037e2b9fcc2ee702dc2d1f399cc9284fadcaf8b: Status 404 returned error can't find the container with id c0b288415af848ae6563a8097037e2b9fcc2ee702dc2d1f399cc9284fadcaf8b
Jul 15 05:03:18.722581 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:18.722529    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:18.757971 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:18.757937    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn" event={"ID":"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a","Type":"ContainerStarted","Data":"bc527d96a202b55c42cb48284d733a6e3f5e839d78145fef3d2926b756aeeb24"}
Jul 15 05:03:18.758152 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:18.757997    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn" event={"ID":"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a","Type":"ContainerStarted","Data":"c0b288415af848ae6563a8097037e2b9fcc2ee702dc2d1f399cc9284fadcaf8b"}
Jul 15 05:03:18.758152 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:18.758046    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:18.777776 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:18.777723    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn" podStartSLOduration=16.777706939 podStartE2EDuration="16.777706939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:03:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:18.776741763 +0000 UTC m=+281.380294382" watchObservedRunningTime="2026-07-15 05:03:18.777706939 +0000 UTC m=+281.381259576"
Jul 15 05:03:18.922481 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:18.922444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:19.020456 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:19.020362    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:19.320320 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:19.320222    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:19.436640 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:19.436594    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bhmbw" podUID="be469964-3786-4205-a37b-5a0198eed10d"
Jul 15 05:03:19.761410 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:19.761361    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bhmbw"
Jul 15 05:03:19.974513 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:19.974475    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:20.123738 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:20.123665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:21.943850 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:21.943817    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:22.045138 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:22.045104    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:22.145861 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:22.145830    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:22.522799 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:22.522767    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:22.724084 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:22.724052    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:22.957116 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:22.957089    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:23.127508 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:23.127475    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:23.283228 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:23.283140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 05:03:23.285445 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:23.285415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/be469964-3786-4205-a37b-5a0198eed10d-metrics-tls\") pod \"dns-default-bhmbw\" (UID: \"be469964-3786-4205-a37b-5a0198eed10d\") " pod="openshift-dns/dns-default-bhmbw"
Jul 15 05:03:23.364859 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:23.364828    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-crtzp\""
Jul 15 05:03:23.373236 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:23.373206    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bhmbw"
Jul 15 05:03:23.499544 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:23.499511    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bhmbw"]
Jul 15 05:03:23.502748 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:03:23.502722    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe469964_3786_4205_a37b_5a0198eed10d.slice/crio-d74c4e19dc00a81d7717dafd36f2666cb9fe5fb1984deef26cc0743563401151 WatchSource:0}: Error finding container d74c4e19dc00a81d7717dafd36f2666cb9fe5fb1984deef26cc0743563401151: Status 404 returned error can't find the container with id d74c4e19dc00a81d7717dafd36f2666cb9fe5fb1984deef26cc0743563401151
Jul 15 05:03:23.776006 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:23.775975    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bhmbw" event={"ID":"be469964-3786-4205-a37b-5a0198eed10d","Type":"ContainerStarted","Data":"d74c4e19dc00a81d7717dafd36f2666cb9fe5fb1984deef26cc0743563401151"}
Jul 15 05:03:23.990348 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:23.990305    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:23.990794 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:23.990410    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:23.991001 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:23.990982    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a222e79e-cdf0-4e4f-aa18-3a30cbede232-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:23.993088 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:23.993061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a222e79e-cdf0-4e4f-aa18-3a30cbede232-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dqv5\" (UID: \"a222e79e-cdf0-4e4f-aa18-3a30cbede232\") " pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:24.010025 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:24.009994    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2dqv5"
Jul 15 05:03:24.178146 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:24.178111    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2dqv5"]
Jul 15 05:03:24.181101 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:03:24.181077    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda222e79e_cdf0_4e4f_aa18_3a30cbede232.slice/crio-5b3fb85d30512358569cc6d9fbce0fbb503e8e5eb19c74a65a7b1f7dab5c1e21 WatchSource:0}: Error finding container 5b3fb85d30512358569cc6d9fbce0fbb503e8e5eb19c74a65a7b1f7dab5c1e21: Status 404 returned error can't find the container with id 5b3fb85d30512358569cc6d9fbce0fbb503e8e5eb19c74a65a7b1f7dab5c1e21
Jul 15 05:03:24.781106 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:24.781075    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2dqv5" event={"ID":"a222e79e-cdf0-4e4f-aa18-3a30cbede232","Type":"ContainerStarted","Data":"5b3fb85d30512358569cc6d9fbce0fbb503e8e5eb19c74a65a7b1f7dab5c1e21"}
Jul 15 05:03:25.786207 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:25.786171    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bhmbw" event={"ID":"be469964-3786-4205-a37b-5a0198eed10d","Type":"ContainerStarted","Data":"e95b5e9c13fefc018d28a201e53d87b66f49edb646abd453bb7b0418ff66efee"}
Jul 15 05:03:25.786693 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:25.786232    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bhmbw" event={"ID":"be469964-3786-4205-a37b-5a0198eed10d","Type":"ContainerStarted","Data":"753019faba69a1a59b2ddbaa45be71985e58fca2b4252098e898f66ef4c7769d"}
Jul 15 05:03:25.786693 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:25.786295    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bhmbw"
Jul 15 05:03:25.804403 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:25.804331    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bhmbw" podStartSLOduration=251.351088734 podStartE2EDuration="4m12.804314495s" totalImagesPullingTime="1.453225761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:25.803670902 +0000 UTC m=+288.407223510" watchObservedRunningTime="2026-07-15 05:03:25.804314495 +0000 UTC m=+288.407867103"
Jul 15 05:03:28.603944 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:28.603911    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:28.658339 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:28.658309    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:28.797420 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:28.797314    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2dqv5" event={"ID":"a222e79e-cdf0-4e4f-aa18-3a30cbede232","Type":"ContainerStarted","Data":"be157d4936a4b128440757ad6279fdb18d97613ee69df2ebb6ed11c1de967ea2"}
Jul 15 05:03:29.677606 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:29.677577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:29.778575 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:29.778542    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:29.802345 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:29.802307    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2dqv5" event={"ID":"a222e79e-cdf0-4e4f-aa18-3a30cbede232","Type":"ContainerStarted","Data":"5712ef260b3ec5512abc9112591dbeaf0adeafdb3a6e967b302ad71bc774d2d9"}
Jul 15 05:03:29.947579 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:29.947545    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:30.122312 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:30.122271    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:30.517505 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:30.517473    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:30.580101 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:30.580068    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:30.950809 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:30.950782    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:31.004418 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.004386    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:31.157263 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.157230    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:31.257831 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.257692    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:31.525670 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.525578    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c78dbcc8c-gjttn"]
Jul 15 05:03:31.610328 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.610294    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56894f6479-5p4v4"]
Jul 15 05:03:31.625249 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.625223    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.640809 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.640777    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56894f6479-5p4v4"]
Jul 15 05:03:31.653633 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.653605    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9c5193cb-c715-4860-a390-3e3f228be090-image-registry-private-configuration\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.653803 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.653642    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfphh\" (UniqueName: \"kubernetes.io/projected/9c5193cb-c715-4860-a390-3e3f228be090-kube-api-access-wfphh\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.653803 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.653665    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9c5193cb-c715-4860-a390-3e3f228be090-installation-pull-secrets\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.653803 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.653679    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c5193cb-c715-4860-a390-3e3f228be090-bound-sa-token\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.653803 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.653695    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9c5193cb-c715-4860-a390-3e3f228be090-registry-certificates\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.653803 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.653780    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9c5193cb-c715-4860-a390-3e3f228be090-trusted-ca\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.654095 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.654075    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9c5193cb-c715-4860-a390-3e3f228be090-ca-trust-extracted\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.654154 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.654117    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c5193cb-c715-4860-a390-3e3f228be090-registry-tls\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.754953 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.754920    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9c5193cb-c715-4860-a390-3e3f228be090-image-registry-private-configuration\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.755106 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.754961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfphh\" (UniqueName: \"kubernetes.io/projected/9c5193cb-c715-4860-a390-3e3f228be090-kube-api-access-wfphh\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.755106 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.755062    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9c5193cb-c715-4860-a390-3e3f228be090-installation-pull-secrets\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.755106 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.755088    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c5193cb-c715-4860-a390-3e3f228be090-bound-sa-token\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.755264 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.755114    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9c5193cb-c715-4860-a390-3e3f228be090-registry-certificates\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.755315 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.755270    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9c5193cb-c715-4860-a390-3e3f228be090-trusted-ca\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.755365 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.755312    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9c5193cb-c715-4860-a390-3e3f228be090-ca-trust-extracted\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.755365 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.755358    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c5193cb-c715-4860-a390-3e3f228be090-registry-tls\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.756020 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.755994    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9c5193cb-c715-4860-a390-3e3f228be090-registry-certificates\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.756140 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.756061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9c5193cb-c715-4860-a390-3e3f228be090-ca-trust-extracted\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.756648 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.756624    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9c5193cb-c715-4860-a390-3e3f228be090-trusted-ca\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.757484 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.757455    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9c5193cb-c715-4860-a390-3e3f228be090-installation-pull-secrets\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.757594 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.757458    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9c5193cb-c715-4860-a390-3e3f228be090-image-registry-private-configuration\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.757893 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.757875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c5193cb-c715-4860-a390-3e3f228be090-registry-tls\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.766305 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.766280    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c5193cb-c715-4860-a390-3e3f228be090-bound-sa-token\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.766502 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.766485    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfphh\" (UniqueName: \"kubernetes.io/projected/9c5193cb-c715-4860-a390-3e3f228be090-kube-api-access-wfphh\") pod \"image-registry-56894f6479-5p4v4\" (UID: \"9c5193cb-c715-4860-a390-3e3f228be090\") " pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:31.811445 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.811325    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2dqv5" event={"ID":"a222e79e-cdf0-4e4f-aa18-3a30cbede232","Type":"ContainerStarted","Data":"cee28b1edb40fa7ff719c1591e3a9ac3d1ff7801c760cf698980f35930295029"}
Jul 15 05:03:31.835142 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.835088    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2dqv5" podStartSLOduration=16.429265882 podStartE2EDuration="23.835070577s" totalImagesPullingTime="7.405804695s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 05:03:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:31.834538144 +0000 UTC m=+294.438090752" watchObservedRunningTime="2026-07-15 05:03:31.835070577 +0000 UTC m=+294.438623185"
Jul 15 05:03:31.942077 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:31.942039    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:32.069071 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:32.068992    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56894f6479-5p4v4"]
Jul 15 05:03:32.072156 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:03:32.072132    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c5193cb_c715_4860_a390_3e3f228be090.slice/crio-b595b47a7756f77d0ff82152e77d997a0065b7efedc7100906669b517dae004c WatchSource:0}: Error finding container b595b47a7756f77d0ff82152e77d997a0065b7efedc7100906669b517dae004c: Status 404 returned error can't find the container with id b595b47a7756f77d0ff82152e77d997a0065b7efedc7100906669b517dae004c
Jul 15 05:03:32.339546 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:32.339442    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:32.400804 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:32.400767    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:32.816133 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:32.816089    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56894f6479-5p4v4" event={"ID":"9c5193cb-c715-4860-a390-3e3f228be090","Type":"ContainerStarted","Data":"a0646f8c5c0631d129ab5dc8639e13d3e8e82ad46353f865e8049cf1a9c2e211"}
Jul 15 05:03:32.816299 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:32.816147    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56894f6479-5p4v4" event={"ID":"9c5193cb-c715-4860-a390-3e3f228be090","Type":"ContainerStarted","Data":"b595b47a7756f77d0ff82152e77d997a0065b7efedc7100906669b517dae004c"}
Jul 15 05:03:32.816299 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:32.816242    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:32.836136 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:32.836089    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56894f6479-5p4v4" podStartSLOduration=1.836073879 podStartE2EDuration="1.836073879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:03:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:32.834699328 +0000 UTC m=+295.438251936" watchObservedRunningTime="2026-07-15 05:03:32.836073879 +0000 UTC m=+295.439626623"
Jul 15 05:03:33.350071 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:33.350039    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:33.400689 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:33.400642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:33.839855 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:33.839810    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9vxv5" podUID="30b62c6b-b454-403b-b1be-4fd921f4a5e4"
Jul 15 05:03:34.478694 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:34.478652    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:34.481011 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:34.480989    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6169dacf-0931-4554-b48e-5da0d8b36ddd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xwctc\" (UID: \"6169dacf-0931-4554-b48e-5da0d8b36ddd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:34.687322 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:34.687285    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"
Jul 15 05:03:34.812560 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:34.812532    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc"]
Jul 15 05:03:34.815864 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:03:34.815839    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6169dacf_0931_4554_b48e_5da0d8b36ddd.slice/crio-5de6d09fd9b19ed8438cbde6a083e1ccf4156e1b901d0ec63ccae3e9a4875ffa WatchSource:0}: Error finding container 5de6d09fd9b19ed8438cbde6a083e1ccf4156e1b901d0ec63ccae3e9a4875ffa: Status 404 returned error can't find the container with id 5de6d09fd9b19ed8438cbde6a083e1ccf4156e1b901d0ec63ccae3e9a4875ffa
Jul 15 05:03:34.823402 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:34.823357    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc" event={"ID":"6169dacf-0931-4554-b48e-5da0d8b36ddd","Type":"ContainerStarted","Data":"5de6d09fd9b19ed8438cbde6a083e1ccf4156e1b901d0ec63ccae3e9a4875ffa"}
Jul 15 05:03:35.791979 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:35.791948    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bhmbw"
Jul 15 05:03:36.791221 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.791185    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bd7c8746c-m5hgw"]
Jul 15 05:03:36.794326 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.794304    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:36.797023 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.797003    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 15 05:03:36.797251 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.797234    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 15 05:03:36.798343 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.798323    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5pgb9\""
Jul 15 05:03:36.798480 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.798385    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 15 05:03:36.798480 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.798425    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 15 05:03:36.798602 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.798587    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 15 05:03:36.798678 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.798657    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 05:03:36.798678 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.798666    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 05:03:36.806690 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.806669    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bd7c8746c-m5hgw"]
Jul 15 05:03:36.899767 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.899728    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-config\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:36.899767 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.899770    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-service-ca\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:36.899987 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.899789    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-serving-cert\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:36.899987 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.899804    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-oauth-serving-cert\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:36.899987 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.899826    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr8jr\" (UniqueName: \"kubernetes.io/projected/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-kube-api-access-gr8jr\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:36.899987 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:36.899850    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-oauth-config\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.000487 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.000447    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-service-ca\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.000487 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.000490    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-serving-cert\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.000725 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.000528    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-oauth-serving-cert\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.000725 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.000553    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr8jr\" (UniqueName: \"kubernetes.io/projected/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-kube-api-access-gr8jr\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.000725 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.000586    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-oauth-config\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.000725 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.000677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-config\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.001212 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.001185    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-oauth-serving-cert\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.001329 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.001187    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-service-ca\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.001329 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.001280    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-config\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.002927 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.002897    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-oauth-config\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.003030 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.002989    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-serving-cert\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.008890 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.008859    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr8jr\" (UniqueName: \"kubernetes.io/projected/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-kube-api-access-gr8jr\") pod \"console-5bd7c8746c-m5hgw\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") " pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.104220 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.104129    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:37.251193 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.251159    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bd7c8746c-m5hgw"]
Jul 15 05:03:37.253983 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:03:37.253957    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3609dafe_07cf_47f2_9bfb_f2a50c13e4e6.slice/crio-cbd0f6b262bdc77e6b463d4309188c6bde28d4760e186fbd1d1cea6638b3440f WatchSource:0}: Error finding container cbd0f6b262bdc77e6b463d4309188c6bde28d4760e186fbd1d1cea6638b3440f: Status 404 returned error can't find the container with id cbd0f6b262bdc77e6b463d4309188c6bde28d4760e186fbd1d1cea6638b3440f
Jul 15 05:03:37.835448 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:37.835414    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bd7c8746c-m5hgw" event={"ID":"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6","Type":"ContainerStarted","Data":"cbd0f6b262bdc77e6b463d4309188c6bde28d4760e186fbd1d1cea6638b3440f"}
Jul 15 05:03:38.124869 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:38.124801    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:03:38.124869 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:38.124855    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:03:38.127283 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:38.127262    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 05:03:38.482115 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:38.482076    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:38.840766 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:38.840650    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc" event={"ID":"6169dacf-0931-4554-b48e-5da0d8b36ddd","Type":"ContainerStarted","Data":"955e760c9a2d3a4a17893c63459c9f2580a1ff5c9b747b7c75f2525e2a433ca8"}
Jul 15 05:03:38.858941 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:38.858885    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xwctc" podStartSLOduration=33.528241517 podStartE2EDuration="36.858867383s" totalImagesPullingTime="3.330625866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:38.85676502 +0000 UTC m=+301.460317620" watchObservedRunningTime="2026-07-15 05:03:38.858867383 +0000 UTC m=+301.462419991"
Jul 15 05:03:40.663904 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:40.663878    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:40.849358 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:40.849320    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bd7c8746c-m5hgw" event={"ID":"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6","Type":"ContainerStarted","Data":"5339b63eee5ae755156575aa8ccdf81032a2872f4e75d2e5a5c9d1014fe0c2f6"}
Jul 15 05:03:40.868843 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:40.868794    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bd7c8746c-m5hgw" podStartSLOduration=1.5005102479999999 podStartE2EDuration="4.868776974s" totalImagesPullingTime="3.368266726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:40.867945545 +0000 UTC m=+303.471498154" watchObservedRunningTime="2026-07-15 05:03:40.868776974 +0000 UTC m=+303.472329582"
Jul 15 05:03:41.530947 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:41.530919    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:43.674130 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:43.674098    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:43.749388 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:43.749345    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:43.884720 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:43.884693    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:43.986704 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:43.986677    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:44.085920 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:44.085887    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:46.352731 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:46.352697    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:47.104514 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:47.104486    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:47.104693 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:47.104541    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:47.108860 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:47.108831    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:47.841078 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:47.841047    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 05:03:47.876453 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:47.876427    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:03:49.756553 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:49.756518    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:49.991085 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:49.991058    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:52.835833 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:52.835803    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:52.920067 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:52.920039    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:53.022598 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:53.022567    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:53.823617 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:53.823589    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56894f6479-5p4v4"
Jul 15 05:03:53.831849 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:53.831819    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:53.949864 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:53.949827    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:54.053943 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:54.053914    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:54.157008 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:54.156927    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:54.258883 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:54.258853    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:54.552040 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:54.552010    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:54.611955 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:54.611923    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:55.819414 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:55.819344    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:55.921473 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:55.921442    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:56.043874 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.043843    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:56.078592 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.078494    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-n88cb"]
Jul 15 05:03:56.084530 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.084499    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.088915 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.088881    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 05:03:56.089077 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.088891    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 05:03:56.089077 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.088891    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fl7xt\""
Jul 15 05:03:56.089500 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.089482    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 05:03:56.099745 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.099722    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 05:03:56.142111 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.142075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:56.165186 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.165148    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-tls\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.165420 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.165253    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-accelerators-collector-config\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.165420 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.165316    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4zjj\" (UniqueName: \"kubernetes.io/projected/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-kube-api-access-r4zjj\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.165420 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.165340    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-textfile\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.165420 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.165361    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-metrics-client-ca\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.165420 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.165392    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-root\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.165679 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.165426    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-sys\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.165679 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.165470    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.165679 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.165499    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-wtmp\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.241453 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.241417    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:56.266675 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.266632    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.266811 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.266686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-wtmp\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.266811 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.266742    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-tls\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.266811 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.266785    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-accelerators-collector-config\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.266965 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.266825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4zjj\" (UniqueName: \"kubernetes.io/projected/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-kube-api-access-r4zjj\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.266965 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.266853    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-textfile\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.266965 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.266884    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-metrics-client-ca\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.266965 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.266895    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-wtmp\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.266965 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.266910    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-root\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.266965 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.266955    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-sys\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.267241 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.266960    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-root\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.267241 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.267042    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-sys\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.267333 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:56.267257    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 15 05:03:56.267333 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.267274    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-textfile\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.267446 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:56.267333    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-tls podName:8c760c2f-4a4b-4d8d-990f-325ad5eb6846 nodeName:}" failed. No retries permitted until 2026-07-15 05:03:56.767313838 +0000 UTC m=+319.370866433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-tls") pod "node-exporter-n88cb" (UID: "8c760c2f-4a4b-4d8d-990f-325ad5eb6846") : secret "node-exporter-tls" not found
Jul 15 05:03:56.267592 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.267572    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-metrics-client-ca\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.267644 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.267576    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-accelerators-collector-config\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.269255 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.269236    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.277687 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.277661    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4zjj\" (UniqueName: \"kubernetes.io/projected/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-kube-api-access-r4zjj\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.357007 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.356909    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:56.419902 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.419873    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:56.443595 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.443560    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bd7c8746c-m5hgw"]
Jul 15 05:03:56.487909 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.487881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:56.549781 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.549734    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn" podUID="83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" containerName="registry" containerID="cri-o://bc527d96a202b55c42cb48284d733a6e3f5e839d78145fef3d2926b756aeeb24" gracePeriod=30
Jul 15 05:03:56.597576 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.597536    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:56.771944 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.771916    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-tls\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.774201 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.774167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c760c2f-4a4b-4d8d-990f-325ad5eb6846-node-exporter-tls\") pod \"node-exporter-n88cb\" (UID: \"8c760c2f-4a4b-4d8d-990f-325ad5eb6846\") " pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:56.788140 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.788111    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:56.873110 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.873076    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-ca-trust-extracted\") pod \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") "
Jul 15 05:03:56.873110 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.873109    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls\") pod \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") "
Jul 15 05:03:56.873632 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.873128    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-bound-sa-token\") pod \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") "
Jul 15 05:03:56.873632 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.873153    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-trusted-ca\") pod \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") "
Jul 15 05:03:56.873632 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.873182    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-image-registry-private-configuration\") pod \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") "
Jul 15 05:03:56.873632 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.873215    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lw49j\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-kube-api-access-lw49j\") pod \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") "
Jul 15 05:03:56.873632 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.873232    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-certificates\") pod \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") "
Jul 15 05:03:56.873632 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.873258    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-installation-pull-secrets\") pod \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\" (UID: \"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a\") "
Jul 15 05:03:56.873907 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.873824    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" (UID: "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:03:56.873967 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.873938    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" (UID: "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:03:56.876004 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.875970    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" (UID: "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:03:56.876119 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.875980    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-kube-api-access-lw49j" (OuterVolumeSpecName: "kube-api-access-lw49j") pod "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" (UID: "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a"). InnerVolumeSpecName "kube-api-access-lw49j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:03:56.876228 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.876210    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" (UID: "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:03:56.876299 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.876234    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" (UID: "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:03:56.876518 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.876494    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" (UID: "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:03:56.881982 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.881959    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" (UID: "83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:03:56.900944 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.900912    2562 generic.go:356] "Generic (PLEG): container finished" podID="83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" containerID="bc527d96a202b55c42cb48284d733a6e3f5e839d78145fef3d2926b756aeeb24" exitCode=0
Jul 15 05:03:56.901088 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.900960    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn"
Jul 15 05:03:56.901088 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.900987    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn" event={"ID":"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a","Type":"ContainerDied","Data":"bc527d96a202b55c42cb48284d733a6e3f5e839d78145fef3d2926b756aeeb24"}
Jul 15 05:03:56.901088 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.901030    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c78dbcc8c-gjttn" event={"ID":"83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a","Type":"ContainerDied","Data":"c0b288415af848ae6563a8097037e2b9fcc2ee702dc2d1f399cc9284fadcaf8b"}
Jul 15 05:03:56.901088 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.901064    2562 scope.go:117] "RemoveContainer" containerID="bc527d96a202b55c42cb48284d733a6e3f5e839d78145fef3d2926b756aeeb24"
Jul 15 05:03:56.909012 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.908992    2562 scope.go:117] "RemoveContainer" containerID="bc527d96a202b55c42cb48284d733a6e3f5e839d78145fef3d2926b756aeeb24"
Jul 15 05:03:56.909296 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:03:56.909274    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc527d96a202b55c42cb48284d733a6e3f5e839d78145fef3d2926b756aeeb24\": container with ID starting with bc527d96a202b55c42cb48284d733a6e3f5e839d78145fef3d2926b756aeeb24 not found: ID does not exist" containerID="bc527d96a202b55c42cb48284d733a6e3f5e839d78145fef3d2926b756aeeb24"
Jul 15 05:03:56.909348 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.909304    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc527d96a202b55c42cb48284d733a6e3f5e839d78145fef3d2926b756aeeb24"} err="failed to get container status \"bc527d96a202b55c42cb48284d733a6e3f5e839d78145fef3d2926b756aeeb24\": rpc error: code = NotFound desc = could not find container \"bc527d96a202b55c42cb48284d733a6e3f5e839d78145fef3d2926b756aeeb24\": container with ID starting with bc527d96a202b55c42cb48284d733a6e3f5e839d78145fef3d2926b756aeeb24 not found: ID does not exist"
Jul 15 05:03:56.943884 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.943804    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c78dbcc8c-gjttn"]
Jul 15 05:03:56.955069 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.955044    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-c78dbcc8c-gjttn"]
Jul 15 05:03:56.974167 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.974142    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-ca-trust-extracted\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:03:56.974264 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.974169    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-tls\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:03:56.974264 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.974179    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-bound-sa-token\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:03:56.974264 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.974187    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-trusted-ca\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:03:56.974264 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.974197    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-image-registry-private-configuration\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:03:56.974264 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.974205    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw49j\" (UniqueName: \"kubernetes.io/projected/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-kube-api-access-lw49j\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:03:56.974264 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.974213    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-registry-certificates\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:03:56.974264 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.974221    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a-installation-pull-secrets\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:03:56.995260 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:56.995237    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n88cb"
Jul 15 05:03:57.008018 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:03:57.007992    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c760c2f_4a4b_4d8d_990f_325ad5eb6846.slice/crio-bb8580938a7799b62b285f232031caa726d6ef0e5402be9e781137eebf52bc9b WatchSource:0}: Error finding container bb8580938a7799b62b285f232031caa726d6ef0e5402be9e781137eebf52bc9b: Status 404 returned error can't find the container with id bb8580938a7799b62b285f232031caa726d6ef0e5402be9e781137eebf52bc9b
Jul 15 05:03:57.009698 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:57.009683    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 05:03:57.063248 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:57.063208    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:57.159956 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:57.159906    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:57.278555 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:57.278516    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:57.556783 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:57.556682    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:57.844756 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:57.844677    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" path="/var/lib/kubelet/pods/83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a/volumes"
Jul 15 05:03:57.905659 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:57.905625    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n88cb" event={"ID":"8c760c2f-4a4b-4d8d-990f-325ad5eb6846","Type":"ContainerStarted","Data":"bb8580938a7799b62b285f232031caa726d6ef0e5402be9e781137eebf52bc9b"}
Jul 15 05:03:58.612139 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:58.612104    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:58.708214 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:58.708177    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:58.911132 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:58.911047    2562 generic.go:356] "Generic (PLEG): container finished" podID="8c760c2f-4a4b-4d8d-990f-325ad5eb6846" containerID="900516ef6b24e1dfd419fe82c2443dfd690096b11883c362136e6f355ee3abc9" exitCode=0
Jul 15 05:03:58.911132 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:58.911123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n88cb" event={"ID":"8c760c2f-4a4b-4d8d-990f-325ad5eb6846","Type":"ContainerDied","Data":"900516ef6b24e1dfd419fe82c2443dfd690096b11883c362136e6f355ee3abc9"}
Jul 15 05:03:59.025941 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:59.025902    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:59.127970 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:59.127944    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:59.310307 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:59.310271    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:59.657913 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:59.657823    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:03:59.917872 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:59.917788    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n88cb" event={"ID":"8c760c2f-4a4b-4d8d-990f-325ad5eb6846","Type":"ContainerStarted","Data":"04493051155c35334acb74cd27218d8ae9237fb2df992b31f1cc386946690600"}
Jul 15 05:03:59.917872 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:59.917841    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n88cb" event={"ID":"8c760c2f-4a4b-4d8d-990f-325ad5eb6846","Type":"ContainerStarted","Data":"64d4bea43912a47a72ccc6d6eff61b27c7bfe9acc1e50a7bc445029b89081d49"}
Jul 15 05:03:59.941575 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:03:59.941524    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-n88cb" podStartSLOduration=2.78749876 podStartE2EDuration="3.941508586s" totalImagesPullingTime="1.091712983s" totalInitContainerRuntime="62.296843ms" isStatefulPod=true podCreationTimestamp="2026-07-15 05:03:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:03:59.939535411 +0000 UTC m=+322.543088019" watchObservedRunningTime="2026-07-15 05:03:59.941508586 +0000 UTC m=+322.545061194"
Jul 15 05:04:00.866298 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:00.866269    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-786845857f-g22rk"]
Jul 15 05:04:00.866578 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:00.866564    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" containerName="registry"
Jul 15 05:04:00.866627 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:00.866580    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" containerName="registry"
Jul 15 05:04:00.866664 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:00.866628    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="83b3bfcb-ecfd-49ec-b48c-b4e6fc95dd2a" containerName="registry"
Jul 15 05:04:00.869286 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:00.869271    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:00.877196 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:00.877168    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 15 05:04:00.881974 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:00.881948    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:00.883000 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:00.882978    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-786845857f-g22rk"]
Jul 15 05:04:00.973873 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:00.973846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:01.007132 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.007094    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-service-ca\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.007300 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.007147    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-serving-cert\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.007300 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.007274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-oauth-config\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.007459 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.007308    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5897f\" (UniqueName: \"kubernetes.io/projected/e4fa0dbe-b1b8-411e-b333-d694b1f39248-kube-api-access-5897f\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.007459 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.007406    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-trusted-ca-bundle\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.007568 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.007484    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-config\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.007568 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.007524    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-oauth-serving-cert\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.073398 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.073348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:01.108537 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.108489    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-service-ca\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.108537 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.108534    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-serving-cert\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.108789 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.108574    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-oauth-config\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.108789 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.108706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5897f\" (UniqueName: \"kubernetes.io/projected/e4fa0dbe-b1b8-411e-b333-d694b1f39248-kube-api-access-5897f\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.108897 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.108827    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-trusted-ca-bundle\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.108897 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.108878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-config\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.109004 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.108928    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-oauth-serving-cert\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.109447 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.109353    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-service-ca\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.109661 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.109624    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-config\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.109847 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.109825    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-oauth-serving-cert\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.110018 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.110000    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-trusted-ca-bundle\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.111161 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.111136    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-oauth-config\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.111429 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.111412    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-serving-cert\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.116937 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.116875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5897f\" (UniqueName: \"kubernetes.io/projected/e4fa0dbe-b1b8-411e-b333-d694b1f39248-kube-api-access-5897f\") pod \"console-786845857f-g22rk\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") " pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.125537 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.125514    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:01.178925 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.178890    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:01.239733 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.239699    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:01.314966 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.314935    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-786845857f-g22rk"]
Jul 15 05:04:01.317870 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:04:01.317834    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4fa0dbe_b1b8_411e_b333_d694b1f39248.slice/crio-731c5fb7e5b6206ffcc3b324f5d8b540211b8d3cd9bb799ba8e052db44f475f4 WatchSource:0}: Error finding container 731c5fb7e5b6206ffcc3b324f5d8b540211b8d3cd9bb799ba8e052db44f475f4: Status 404 returned error can't find the container with id 731c5fb7e5b6206ffcc3b324f5d8b540211b8d3cd9bb799ba8e052db44f475f4
Jul 15 05:04:01.929456 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.926201    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-786845857f-g22rk" event={"ID":"e4fa0dbe-b1b8-411e-b333-d694b1f39248","Type":"ContainerStarted","Data":"f9efc9a09ec7f4d0ca0b93d198573bc57b5fa58cec695c99cd0d025fc742111a"}
Jul 15 05:04:01.929456 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.926300    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-786845857f-g22rk" event={"ID":"e4fa0dbe-b1b8-411e-b333-d694b1f39248","Type":"ContainerStarted","Data":"731c5fb7e5b6206ffcc3b324f5d8b540211b8d3cd9bb799ba8e052db44f475f4"}
Jul 15 05:04:01.945893 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:01.945841    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-786845857f-g22rk" podStartSLOduration=1.945822626 podStartE2EDuration="1.945822626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:04:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:04:01.945585151 +0000 UTC m=+324.549137762" watchObservedRunningTime="2026-07-15 05:04:01.945822626 +0000 UTC m=+324.549375235"
Jul 15 05:04:02.293955 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.293921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:02.303934 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.303906    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 05:04:02.365847 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.365809    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 05:04:02.366009 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.365960    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.368884 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.368859    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 05:04:02.369028 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.368889    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 05:04:02.369028 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.369023    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 05:04:02.372828 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.372791    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 05:04:02.372982 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.372961    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 05:04:02.373051 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.373036    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jm5l9\""
Jul 15 05:04:02.373112 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.373061    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 05:04:02.373112 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.373102    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 05:04:02.373208 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.373169    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 05:04:02.373256 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.373216    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 05:04:02.373256 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.373216    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4rkup20obssa5\""
Jul 15 05:04:02.373256 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.373245    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 05:04:02.373418 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.373402    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 05:04:02.373477 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.373420    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 05:04:02.373477 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.373423    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 05:04:02.374137 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.374119    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 05:04:02.375355 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.375331    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 05:04:02.522226 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522188    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddd0578b-11d7-4599-babd-353350e5d10c-config-out\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522226 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522227    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522500 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522269    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522500 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-web-config\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522500 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522318    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522500 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522334    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522500 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522431    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522500 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522459    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522500 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522734 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522529    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522734 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-config\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522734 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522569    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddd0578b-11d7-4599-babd-353350e5d10c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522734 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522591    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t99tq\" (UniqueName: \"kubernetes.io/projected/ddd0578b-11d7-4599-babd-353350e5d10c-kube-api-access-t99tq\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522734 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522611    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522734 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522632    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522734 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.522734 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522730    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.523014 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522756    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.523014 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522773    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.523014 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.522867    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.623936 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.623834    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.623936 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.623880    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddd0578b-11d7-4599-babd-353350e5d10c-config-out\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.623936 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.623904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624179 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.623940    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624179 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.623960    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-web-config\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624179 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.623982    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624179 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624004    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624179 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624179 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624179 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624179 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624179 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624159    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-config\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624618 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624184    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddd0578b-11d7-4599-babd-353350e5d10c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624618 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t99tq\" (UniqueName: \"kubernetes.io/projected/ddd0578b-11d7-4599-babd-353350e5d10c-kube-api-access-t99tq\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624618 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624618 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624264    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624618 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624304    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624618 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624618 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624394    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624618 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624423    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624856 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624656    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624890 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624854    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.624890 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.624873    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.625452 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.625423    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.626778 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.626754    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ddd0578b-11d7-4599-babd-353350e5d10c-config-out\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.627228 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.627026    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.627498 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.627468    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.627597 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.627574    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.627667 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.627622    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-web-config\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.627667 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.627622    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.627929 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.627907    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.628231 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.628204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ddd0578b-11d7-4599-babd-353350e5d10c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.628323 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.628270    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.628599 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.628577    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddd0578b-11d7-4599-babd-353350e5d10c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.628944 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.628923    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.629464 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.629435    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-config\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.630211 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.630189    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.630286 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.630233    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.630347 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.630332    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ddd0578b-11d7-4599-babd-353350e5d10c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.635191 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.635175    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t99tq\" (UniqueName: \"kubernetes.io/projected/ddd0578b-11d7-4599-babd-353350e5d10c-kube-api-access-t99tq\") pod \"prometheus-k8s-0\" (UID: \"ddd0578b-11d7-4599-babd-353350e5d10c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.676005 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.675971    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:02.837569 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.837534    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 05:04:02.840568 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:04:02.840541    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd0578b_11d7_4599_babd_353350e5d10c.slice/crio-093288c34851c7594424ea61467876b92a7446a10b9c562c834348fad83a7739 WatchSource:0}: Error finding container 093288c34851c7594424ea61467876b92a7446a10b9c562c834348fad83a7739: Status 404 returned error can't find the container with id 093288c34851c7594424ea61467876b92a7446a10b9c562c834348fad83a7739
Jul 15 05:04:02.931318 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:02.931237    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddd0578b-11d7-4599-babd-353350e5d10c","Type":"ContainerStarted","Data":"093288c34851c7594424ea61467876b92a7446a10b9c562c834348fad83a7739"}
Jul 15 05:04:04.940033 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:04.939996    2562 generic.go:356] "Generic (PLEG): container finished" podID="ddd0578b-11d7-4599-babd-353350e5d10c" containerID="a44f9732c56d74ba3362f0acac8b29cbf026c24d96f2f7ef28d0b0c38724cb5f" exitCode=0
Jul 15 05:04:04.940427 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:04.940077    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddd0578b-11d7-4599-babd-353350e5d10c","Type":"ContainerDied","Data":"a44f9732c56d74ba3362f0acac8b29cbf026c24d96f2f7ef28d0b0c38724cb5f"}
Jul 15 05:04:06.130280 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:06.130248    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:06.185527 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:06.185495    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:08.961406 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:08.961350    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddd0578b-11d7-4599-babd-353350e5d10c","Type":"ContainerStarted","Data":"77010477ade3d65934b374e0a5036c21a7168198ca07193be13a1d84cac48801"}
Jul 15 05:04:08.961820 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:08.961495    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddd0578b-11d7-4599-babd-353350e5d10c","Type":"ContainerStarted","Data":"bab41ff2ca00a009e7a1dab6cb842b5a902e1b47862e2dfe3ade5922e0dfea0f"}
Jul 15 05:04:10.971137 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:10.971103    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddd0578b-11d7-4599-babd-353350e5d10c","Type":"ContainerStarted","Data":"0d98c7c06108a45f8564b3f58864e3c9587e6ee7c9c375faba6cbf7c2dd6f83a"}
Jul 15 05:04:10.971536 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:10.971166    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddd0578b-11d7-4599-babd-353350e5d10c","Type":"ContainerStarted","Data":"0f3af9a98df99e5286b8e8ac6171aaf98acb2f83487c12fbef602acaf2effa29"}
Jul 15 05:04:10.971536 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:10.971192    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddd0578b-11d7-4599-babd-353350e5d10c","Type":"ContainerStarted","Data":"57baa5b44895dfd2cc6ad4a500cd611fc03cd7a906bb69d06378b9b2bfc128bb"}
Jul 15 05:04:10.971536 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:10.971216    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ddd0578b-11d7-4599-babd-353350e5d10c","Type":"ContainerStarted","Data":"8e61036cb9453c7a2370f3fffd08cf6abe0ea1c28b3c49e7e331d04e52f24882"}
Jul 15 05:04:11.002807 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:11.002738    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.7688043479999997 podStartE2EDuration="9.002715683s" totalImagesPullingTime="6.216818646s" totalInitContainerRuntime="17.092689ms" isStatefulPod=false podCreationTimestamp="2026-07-15 05:04:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:04:11.000614139 +0000 UTC m=+333.604166758" watchObservedRunningTime="2026-07-15 05:04:11.002715683 +0000 UTC m=+333.606268294"
Jul 15 05:04:11.179052 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:11.179014    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:11.179229 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:11.179086    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:11.183295 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:11.183271    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:11.978524 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:11.978494    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:04:12.132069 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:12.132037    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:12.232844 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:12.232757    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:12.291273 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:12.291240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:12.676279 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:12.676239    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:04:13.596579 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:13.596544    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:13.652286 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:13.652259    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:21.463936 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.463871    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5bd7c8746c-m5hgw" podUID="3609dafe-07cf-47f2-9bfb-f2a50c13e4e6" containerName="console" containerID="cri-o://5339b63eee5ae755156575aa8ccdf81032a2872f4e75d2e5a5c9d1014fe0c2f6" gracePeriod=15
Jul 15 05:04:21.698114 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.698091    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:04:21.798844 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.798749    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gr8jr\" (UniqueName: \"kubernetes.io/projected/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-kube-api-access-gr8jr\") pod \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") "
Jul 15 05:04:21.798844 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.798795    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-config\") pod \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") "
Jul 15 05:04:21.798844 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.798815    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-service-ca\") pod \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") "
Jul 15 05:04:21.798844 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.798832    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-oauth-config\") pod \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") "
Jul 15 05:04:21.799218 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.799023    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-serving-cert\") pod \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") "
Jul 15 05:04:21.799218 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.799148    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-oauth-serving-cert\") pod \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\" (UID: \"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6\") "
Jul 15 05:04:21.799218 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.799202    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-service-ca" (OuterVolumeSpecName: "service-ca") pod "3609dafe-07cf-47f2-9bfb-f2a50c13e4e6" (UID: "3609dafe-07cf-47f2-9bfb-f2a50c13e4e6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:04:21.799348 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.799210    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-config" (OuterVolumeSpecName: "console-config") pod "3609dafe-07cf-47f2-9bfb-f2a50c13e4e6" (UID: "3609dafe-07cf-47f2-9bfb-f2a50c13e4e6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:04:21.799495 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.799476    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-config\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:04:21.799495 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.799495    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-service-ca\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:04:21.799609 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.799513    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3609dafe-07cf-47f2-9bfb-f2a50c13e4e6" (UID: "3609dafe-07cf-47f2-9bfb-f2a50c13e4e6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:04:21.801163 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.801138    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3609dafe-07cf-47f2-9bfb-f2a50c13e4e6" (UID: "3609dafe-07cf-47f2-9bfb-f2a50c13e4e6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:04:21.801307 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.801291    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3609dafe-07cf-47f2-9bfb-f2a50c13e4e6" (UID: "3609dafe-07cf-47f2-9bfb-f2a50c13e4e6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:04:21.801357 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.801320    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-kube-api-access-gr8jr" (OuterVolumeSpecName: "kube-api-access-gr8jr") pod "3609dafe-07cf-47f2-9bfb-f2a50c13e4e6" (UID: "3609dafe-07cf-47f2-9bfb-f2a50c13e4e6"). InnerVolumeSpecName "kube-api-access-gr8jr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:04:21.900764 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.900726    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-oauth-config\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:04:21.900764 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.900756    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-console-serving-cert\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:04:21.900764 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.900765    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-oauth-serving-cert\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:04:21.900764 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:21.900775    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr8jr\" (UniqueName: \"kubernetes.io/projected/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6-kube-api-access-gr8jr\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:04:22.010808 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:22.010775    2562 generic.go:356] "Generic (PLEG): container finished" podID="3609dafe-07cf-47f2-9bfb-f2a50c13e4e6" containerID="5339b63eee5ae755156575aa8ccdf81032a2872f4e75d2e5a5c9d1014fe0c2f6" exitCode=0
Jul 15 05:04:22.010974 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:22.010822    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bd7c8746c-m5hgw"
Jul 15 05:04:22.010974 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:22.010859    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bd7c8746c-m5hgw" event={"ID":"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6","Type":"ContainerDied","Data":"5339b63eee5ae755156575aa8ccdf81032a2872f4e75d2e5a5c9d1014fe0c2f6"}
Jul 15 05:04:22.010974 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:22.010909    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bd7c8746c-m5hgw" event={"ID":"3609dafe-07cf-47f2-9bfb-f2a50c13e4e6","Type":"ContainerDied","Data":"cbd0f6b262bdc77e6b463d4309188c6bde28d4760e186fbd1d1cea6638b3440f"}
Jul 15 05:04:22.010974 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:22.010930    2562 scope.go:117] "RemoveContainer" containerID="5339b63eee5ae755156575aa8ccdf81032a2872f4e75d2e5a5c9d1014fe0c2f6"
Jul 15 05:04:22.018676 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:22.018660    2562 scope.go:117] "RemoveContainer" containerID="5339b63eee5ae755156575aa8ccdf81032a2872f4e75d2e5a5c9d1014fe0c2f6"
Jul 15 05:04:22.018951 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:04:22.018934    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5339b63eee5ae755156575aa8ccdf81032a2872f4e75d2e5a5c9d1014fe0c2f6\": container with ID starting with 5339b63eee5ae755156575aa8ccdf81032a2872f4e75d2e5a5c9d1014fe0c2f6 not found: ID does not exist" containerID="5339b63eee5ae755156575aa8ccdf81032a2872f4e75d2e5a5c9d1014fe0c2f6"
Jul 15 05:04:22.019011 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:22.018959    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5339b63eee5ae755156575aa8ccdf81032a2872f4e75d2e5a5c9d1014fe0c2f6"} err="failed to get container status \"5339b63eee5ae755156575aa8ccdf81032a2872f4e75d2e5a5c9d1014fe0c2f6\": rpc error: code = NotFound desc = could not find container \"5339b63eee5ae755156575aa8ccdf81032a2872f4e75d2e5a5c9d1014fe0c2f6\": container with ID starting with 5339b63eee5ae755156575aa8ccdf81032a2872f4e75d2e5a5c9d1014fe0c2f6 not found: ID does not exist"
Jul 15 05:04:22.028711 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:22.028690    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bd7c8746c-m5hgw"]
Jul 15 05:04:22.033508 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:22.033486    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bd7c8746c-m5hgw"]
Jul 15 05:04:23.016516 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:23.016485    2562 generic.go:356] "Generic (PLEG): container finished" podID="e2f8ae05-0862-4e90-bde9-1f743802015a" containerID="dd77d0369abac1dfbd90a970996b70a5753352a7a27c72ffcbeefea19864c8a4" exitCode=0
Jul 15 05:04:23.017027 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:23.016563    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2" event={"ID":"e2f8ae05-0862-4e90-bde9-1f743802015a","Type":"ContainerDied","Data":"dd77d0369abac1dfbd90a970996b70a5753352a7a27c72ffcbeefea19864c8a4"}
Jul 15 05:04:23.017027 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:23.016949    2562 scope.go:117] "RemoveContainer" containerID="dd77d0369abac1dfbd90a970996b70a5753352a7a27c72ffcbeefea19864c8a4"
Jul 15 05:04:23.748080 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:23.748052    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p7b4f_c99848fc-5c0a-4482-8535-10bdd21098d2/serve-healthcheck-canary/0.log"
Jul 15 05:04:23.844267 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:23.844231    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3609dafe-07cf-47f2-9bfb-f2a50c13e4e6" path="/var/lib/kubelet/pods/3609dafe-07cf-47f2-9bfb-f2a50c13e4e6/volumes"
Jul 15 05:04:24.022465 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:24.022307    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zkpg2" event={"ID":"e2f8ae05-0862-4e90-bde9-1f743802015a","Type":"ContainerStarted","Data":"d77ff5079b0caf53f4a0b1161b6ef971ae30855e2b89ff5a01cb4f222bbc7d2b"}
Jul 15 05:04:28.595354 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:28.595319    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:28.649739 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:28.649707    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:35.386837 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:35.386743    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:35.487330 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:35.487296    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:35.554657 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:35.554624    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:51.759926 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:51.759874    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 05:04:51.762216 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:51.762187    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30b62c6b-b454-403b-b1be-4fd921f4a5e4-metrics-certs\") pod \"network-metrics-daemon-9vxv5\" (UID: \"30b62c6b-b454-403b-b1be-4fd921f4a5e4\") " pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 05:04:52.044513 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:52.044416    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9bt4r\""
Jul 15 05:04:52.052748 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:52.052719    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vxv5"
Jul 15 05:04:52.182436 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:52.182405    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9vxv5"]
Jul 15 05:04:52.185912 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:04:52.185887    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30b62c6b_b454_403b_b1be_4fd921f4a5e4.slice/crio-ef6ab69254d92e99e02a554a03ed9ae84f89cc3c1baf6b15f8f15a563f2076a2 WatchSource:0}: Error finding container ef6ab69254d92e99e02a554a03ed9ae84f89cc3c1baf6b15f8f15a563f2076a2: Status 404 returned error can't find the container with id ef6ab69254d92e99e02a554a03ed9ae84f89cc3c1baf6b15f8f15a563f2076a2
Jul 15 05:04:52.299007 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:52.298927    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:52.355366 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:52.355332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:53.131755 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:53.131723    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9vxv5" event={"ID":"30b62c6b-b454-403b-b1be-4fd921f4a5e4","Type":"ContainerStarted","Data":"d313e6dd592f8e721ed12e5d062dc9976d24334b66dcc9b4d149e2a2705969bd"}
Jul 15 05:04:53.132173 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:53.131775    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9vxv5" event={"ID":"30b62c6b-b454-403b-b1be-4fd921f4a5e4","Type":"ContainerStarted","Data":"ef6ab69254d92e99e02a554a03ed9ae84f89cc3c1baf6b15f8f15a563f2076a2"}
Jul 15 05:04:54.140304 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:54.140270    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9vxv5" event={"ID":"30b62c6b-b454-403b-b1be-4fd921f4a5e4","Type":"ContainerStarted","Data":"346f74f4d7127f978d74cfb597f19ffe67200dfe040d1a1749db666bfd1c2aca"}
Jul 15 05:04:54.156618 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:54.156567    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9vxv5" podStartSLOduration=375.334591698 podStartE2EDuration="6m16.156550662s" totalImagesPullingTime="821.958964ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 04:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:04:54.156084068 +0000 UTC m=+376.759636694" watchObservedRunningTime="2026-07-15 05:04:54.156550662 +0000 UTC m=+376.760103270"
Jul 15 05:04:54.240689 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:54.240648    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:54.295025 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:54.294947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:57.731663 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:57.731611    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:04:57.787929 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:04:57.787895    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:02.676550 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:02.676515    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:05:02.694962 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:02.694925    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:05:03.187486 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:03.187460    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 05:05:15.341629 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:15.341593    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:15.690650 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:15.690566    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:24.407866 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:24.407829    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:24.472949 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:24.472910    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:30.747619 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:30.747562    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:30.850725 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:30.850690    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:33.279177 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:33.279138    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:34.816871 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:34.816829    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-786845857f-g22rk"]
Jul 15 05:05:42.743994 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:42.743942    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:42.841040 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:42.841009    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:45.981531 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:45.981473    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:46.081905 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:46.081869    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:46.150996 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:46.150949    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:53.922258 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:53.922225    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:53.992093 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:53.992061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:58.608755 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:58.608719    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:58.709502 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:58.709467    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:58.779049 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:58.779023    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:05:59.837240 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:05:59.837156    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-786845857f-g22rk" podUID="e4fa0dbe-b1b8-411e-b333-d694b1f39248" containerName="console" containerID="cri-o://f9efc9a09ec7f4d0ca0b93d198573bc57b5fa58cec695c99cd0d025fc742111a" gracePeriod=15
Jul 15 05:06:00.082543 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.082518    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:06:00.108814 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.108733    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-trusted-ca-bundle\") pod \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") "
Jul 15 05:06:00.108814 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.108779    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-service-ca\") pod \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") "
Jul 15 05:06:00.109034 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.108817    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-config\") pod \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") "
Jul 15 05:06:00.109034 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.108863    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5897f\" (UniqueName: \"kubernetes.io/projected/e4fa0dbe-b1b8-411e-b333-d694b1f39248-kube-api-access-5897f\") pod \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") "
Jul 15 05:06:00.109034 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.108908    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-oauth-serving-cert\") pod \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") "
Jul 15 05:06:00.109034 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.108937    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-serving-cert\") pod \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") "
Jul 15 05:06:00.109034 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.108965    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-oauth-config\") pod \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\" (UID: \"e4fa0dbe-b1b8-411e-b333-d694b1f39248\") "
Jul 15 05:06:00.109340 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.109212    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-service-ca" (OuterVolumeSpecName: "service-ca") pod "e4fa0dbe-b1b8-411e-b333-d694b1f39248" (UID: "e4fa0dbe-b1b8-411e-b333-d694b1f39248"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:06:00.109340 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.109237    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e4fa0dbe-b1b8-411e-b333-d694b1f39248" (UID: "e4fa0dbe-b1b8-411e-b333-d694b1f39248"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:06:00.109460 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.109334    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-config" (OuterVolumeSpecName: "console-config") pod "e4fa0dbe-b1b8-411e-b333-d694b1f39248" (UID: "e4fa0dbe-b1b8-411e-b333-d694b1f39248"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:06:00.109460 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.109445    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e4fa0dbe-b1b8-411e-b333-d694b1f39248" (UID: "e4fa0dbe-b1b8-411e-b333-d694b1f39248"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:06:00.111645 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.111617    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e4fa0dbe-b1b8-411e-b333-d694b1f39248" (UID: "e4fa0dbe-b1b8-411e-b333-d694b1f39248"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:06:00.111875 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.111695    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e4fa0dbe-b1b8-411e-b333-d694b1f39248" (UID: "e4fa0dbe-b1b8-411e-b333-d694b1f39248"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:06:00.111875 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.111754    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4fa0dbe-b1b8-411e-b333-d694b1f39248-kube-api-access-5897f" (OuterVolumeSpecName: "kube-api-access-5897f") pod "e4fa0dbe-b1b8-411e-b333-d694b1f39248" (UID: "e4fa0dbe-b1b8-411e-b333-d694b1f39248"). InnerVolumeSpecName "kube-api-access-5897f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:06:00.210348 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.210314    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-trusted-ca-bundle\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:06:00.210348 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.210342    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-service-ca\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:06:00.210348 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.210351    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-config\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:06:00.210348 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.210359    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5897f\" (UniqueName: \"kubernetes.io/projected/e4fa0dbe-b1b8-411e-b333-d694b1f39248-kube-api-access-5897f\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:06:00.210644 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.210391    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4fa0dbe-b1b8-411e-b333-d694b1f39248-oauth-serving-cert\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:06:00.210644 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.210400    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-serving-cert\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:06:00.210644 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.210410    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4fa0dbe-b1b8-411e-b333-d694b1f39248-console-oauth-config\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:06:00.376654 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.376565    2562 generic.go:356] "Generic (PLEG): container finished" podID="e4fa0dbe-b1b8-411e-b333-d694b1f39248" containerID="f9efc9a09ec7f4d0ca0b93d198573bc57b5fa58cec695c99cd0d025fc742111a" exitCode=0
Jul 15 05:06:00.376654 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.376620    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-786845857f-g22rk"
Jul 15 05:06:00.376654 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.376631    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-786845857f-g22rk" event={"ID":"e4fa0dbe-b1b8-411e-b333-d694b1f39248","Type":"ContainerDied","Data":"f9efc9a09ec7f4d0ca0b93d198573bc57b5fa58cec695c99cd0d025fc742111a"}
Jul 15 05:06:00.377059 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.376676    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-786845857f-g22rk" event={"ID":"e4fa0dbe-b1b8-411e-b333-d694b1f39248","Type":"ContainerDied","Data":"731c5fb7e5b6206ffcc3b324f5d8b540211b8d3cd9bb799ba8e052db44f475f4"}
Jul 15 05:06:00.377059 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.376697    2562 scope.go:117] "RemoveContainer" containerID="f9efc9a09ec7f4d0ca0b93d198573bc57b5fa58cec695c99cd0d025fc742111a"
Jul 15 05:06:00.385218 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.385200    2562 scope.go:117] "RemoveContainer" containerID="f9efc9a09ec7f4d0ca0b93d198573bc57b5fa58cec695c99cd0d025fc742111a"
Jul 15 05:06:00.385522 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:06:00.385500    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9efc9a09ec7f4d0ca0b93d198573bc57b5fa58cec695c99cd0d025fc742111a\": container with ID starting with f9efc9a09ec7f4d0ca0b93d198573bc57b5fa58cec695c99cd0d025fc742111a not found: ID does not exist" containerID="f9efc9a09ec7f4d0ca0b93d198573bc57b5fa58cec695c99cd0d025fc742111a"
Jul 15 05:06:00.385619 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.385528    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9efc9a09ec7f4d0ca0b93d198573bc57b5fa58cec695c99cd0d025fc742111a"} err="failed to get container status \"f9efc9a09ec7f4d0ca0b93d198573bc57b5fa58cec695c99cd0d025fc742111a\": rpc error: code = NotFound desc = could not find container \"f9efc9a09ec7f4d0ca0b93d198573bc57b5fa58cec695c99cd0d025fc742111a\": container with ID starting with f9efc9a09ec7f4d0ca0b93d198573bc57b5fa58cec695c99cd0d025fc742111a not found: ID does not exist"
Jul 15 05:06:00.399688 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.399659    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-786845857f-g22rk"]
Jul 15 05:06:00.404946 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:00.404923    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-786845857f-g22rk"]
Jul 15 05:06:01.843755 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:01.843719    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4fa0dbe-b1b8-411e-b333-d694b1f39248" path="/var/lib/kubelet/pods/e4fa0dbe-b1b8-411e-b333-d694b1f39248/volumes"
Jul 15 05:06:30.465943 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:30.465914    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:06:47.179973 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:47.179931    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:06:48.427747 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:48.427709    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:06:48.527042 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:06:48.527006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:15.360751 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:15.360711    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:16.210691 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:16.210660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:18.985418 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:18.985379    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:19.067551 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:19.067522    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:21.859382 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:21.859337    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:21.954351 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:21.954317    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:22.598663 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:22.598633    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:22.750738 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:22.750702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:23.883324 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:23.883292    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:24.254953 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:24.254921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:30.032365 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:30.032287    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:30.106058 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:30.106025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:32.676059 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:32.676021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:34.197938 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:34.197897    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:34.299607 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:34.299574    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:43.758673 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:43.758635    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:43.862590 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:43.862554    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:45.445274 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:45.445232    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:55.688273 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.688235    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:55.735980 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.735948    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"]
Jul 15 05:07:55.736248 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.736235    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4fa0dbe-b1b8-411e-b333-d694b1f39248" containerName="console"
Jul 15 05:07:55.736301 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.736250    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4fa0dbe-b1b8-411e-b333-d694b1f39248" containerName="console"
Jul 15 05:07:55.736301 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.736269    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3609dafe-07cf-47f2-9bfb-f2a50c13e4e6" containerName="console"
Jul 15 05:07:55.736301 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.736275    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3609dafe-07cf-47f2-9bfb-f2a50c13e4e6" containerName="console"
Jul 15 05:07:55.736438 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.736318    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3609dafe-07cf-47f2-9bfb-f2a50c13e4e6" containerName="console"
Jul 15 05:07:55.736438 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.736328    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4fa0dbe-b1b8-411e-b333-d694b1f39248" containerName="console"
Jul 15 05:07:55.739350 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.739329    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:55.743514 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.743491    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 15 05:07:55.743652 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.743519    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-v8jgz\""
Jul 15 05:07:55.743652 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.743594    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 15 05:07:55.743652 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.743595    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 15 05:07:55.743652 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.743635    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 15 05:07:55.743863 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.743638    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 15 05:07:55.753852 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.753825    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"]
Jul 15 05:07:55.783143 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.783113    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:07:55.867546 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.867510    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca6e8cec-ddb6-44a0-91de-86bad50c8d6a-metrics-cert\") pod \"lws-controller-manager-8fd856859-bmcmr\" (UID: \"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a\") " pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:55.867724 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.867559    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca6e8cec-ddb6-44a0-91de-86bad50c8d6a-cert\") pod \"lws-controller-manager-8fd856859-bmcmr\" (UID: \"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a\") " pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:55.867724 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.867621    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd99m\" (UniqueName: \"kubernetes.io/projected/ca6e8cec-ddb6-44a0-91de-86bad50c8d6a-kube-api-access-zd99m\") pod \"lws-controller-manager-8fd856859-bmcmr\" (UID: \"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a\") " pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:55.867724 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.867693    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ca6e8cec-ddb6-44a0-91de-86bad50c8d6a-manager-config\") pod \"lws-controller-manager-8fd856859-bmcmr\" (UID: \"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a\") " pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:55.968903 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.968868    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ca6e8cec-ddb6-44a0-91de-86bad50c8d6a-manager-config\") pod \"lws-controller-manager-8fd856859-bmcmr\" (UID: \"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a\") " pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:55.969068 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.968925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca6e8cec-ddb6-44a0-91de-86bad50c8d6a-metrics-cert\") pod \"lws-controller-manager-8fd856859-bmcmr\" (UID: \"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a\") " pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:55.969068 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.968968    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca6e8cec-ddb6-44a0-91de-86bad50c8d6a-cert\") pod \"lws-controller-manager-8fd856859-bmcmr\" (UID: \"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a\") " pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:55.969068 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.968992    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd99m\" (UniqueName: \"kubernetes.io/projected/ca6e8cec-ddb6-44a0-91de-86bad50c8d6a-kube-api-access-zd99m\") pod \"lws-controller-manager-8fd856859-bmcmr\" (UID: \"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a\") " pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:55.969585 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.969562    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ca6e8cec-ddb6-44a0-91de-86bad50c8d6a-manager-config\") pod \"lws-controller-manager-8fd856859-bmcmr\" (UID: \"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a\") " pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:55.971480 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.971457    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca6e8cec-ddb6-44a0-91de-86bad50c8d6a-metrics-cert\") pod \"lws-controller-manager-8fd856859-bmcmr\" (UID: \"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a\") " pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:55.971575 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.971533    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca6e8cec-ddb6-44a0-91de-86bad50c8d6a-cert\") pod \"lws-controller-manager-8fd856859-bmcmr\" (UID: \"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a\") " pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:55.978153 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:55.978129    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd99m\" (UniqueName: \"kubernetes.io/projected/ca6e8cec-ddb6-44a0-91de-86bad50c8d6a-kube-api-access-zd99m\") pod \"lws-controller-manager-8fd856859-bmcmr\" (UID: \"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a\") " pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:56.049136 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:56.049092    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:56.190998 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:56.190891    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"]
Jul 15 05:07:56.193758 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:07:56.193734    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca6e8cec_ddb6_44a0_91de_86bad50c8d6a.slice/crio-94e44c8bfb1358aa9498bf02bff92eb0e9d7ae5a1014e040bd3d587a71751968 WatchSource:0}: Error finding container 94e44c8bfb1358aa9498bf02bff92eb0e9d7ae5a1014e040bd3d587a71751968: Status 404 returned error can't find the container with id 94e44c8bfb1358aa9498bf02bff92eb0e9d7ae5a1014e040bd3d587a71751968
Jul 15 05:07:56.785214 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:56.785181    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr" event={"ID":"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a","Type":"ContainerStarted","Data":"94e44c8bfb1358aa9498bf02bff92eb0e9d7ae5a1014e040bd3d587a71751968"}
Jul 15 05:07:59.796699 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:59.796654    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr" event={"ID":"ca6e8cec-ddb6-44a0-91de-86bad50c8d6a","Type":"ContainerStarted","Data":"97f7fccf5de63e4b30ad478def439edd188294cb6fba23b90c642e48a3e89e20"}
Jul 15 05:07:59.797081 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:59.796776    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:07:59.849988 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:07:59.849933    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr" podStartSLOduration=2.053522275 podStartE2EDuration="4.849916683s" totalImagesPullingTime="2.796394408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:07:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:07:59.848441805 +0000 UTC m=+562.451994415" watchObservedRunningTime="2026-07-15 05:07:59.849916683 +0000 UTC m=+562.453469291"
Jul 15 05:08:01.897808 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:01.897773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:02.028578 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:02.028537    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:02.556168 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:02.556128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:03.309033 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.308997    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:03.404694 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.404660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:03.494895 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.494858    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"]
Jul 15 05:08:03.498056 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.498034    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.503072 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.503049    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 05:08:03.503220 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.503047    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 05:08:03.503220 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.503053    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k6dtd\""
Jul 15 05:08:03.503220 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.503050    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 05:08:03.504638 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.504426    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 05:08:03.528467 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.528427    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1780b906-079c-414c-aa53-aea05c06bbbf-tmp\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.528645 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.528478    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1780b906-079c-414c-aa53-aea05c06bbbf-manifests\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.528645 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.528525    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1780b906-079c-414c-aa53-aea05c06bbbf-webhook-cert\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.528645 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.528616    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1780b906-079c-414c-aa53-aea05c06bbbf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.528768 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.528656    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c7ww\" (UniqueName: \"kubernetes.io/projected/1780b906-079c-414c-aa53-aea05c06bbbf-kube-api-access-7c7ww\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.529714 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.529685    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"]
Jul 15 05:08:03.629473 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.629331    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1780b906-079c-414c-aa53-aea05c06bbbf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.629473 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.629467    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c7ww\" (UniqueName: \"kubernetes.io/projected/1780b906-079c-414c-aa53-aea05c06bbbf-kube-api-access-7c7ww\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.629663 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.629496    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1780b906-079c-414c-aa53-aea05c06bbbf-tmp\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.629663 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.629525    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1780b906-079c-414c-aa53-aea05c06bbbf-manifests\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.629663 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.629577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1780b906-079c-414c-aa53-aea05c06bbbf-webhook-cert\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.629947 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.629922    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1780b906-079c-414c-aa53-aea05c06bbbf-manifests\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.631866 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.631839    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1780b906-079c-414c-aa53-aea05c06bbbf-tmp\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.631997 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.631893    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1780b906-079c-414c-aa53-aea05c06bbbf-webhook-cert\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.631997 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.631970    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1780b906-079c-414c-aa53-aea05c06bbbf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.644851 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.644819    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c7ww\" (UniqueName: \"kubernetes.io/projected/1780b906-079c-414c-aa53-aea05c06bbbf-kube-api-access-7c7ww\") pod \"opendatahub-operator-controller-manager-749c75689b-sqgqr\" (UID: \"1780b906-079c-414c-aa53-aea05c06bbbf\") " pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.675298 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.675264    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:03.807962 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.807921    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:03.808932 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.808901    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:03.933885 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.933857    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:03.958569 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:03.958527    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"]
Jul 15 05:08:03.961604 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:08:03.961574    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1780b906_079c_414c_aa53_aea05c06bbbf.slice/crio-d13cc3ca48070aa128bf577738f9a9b81c0defe288179c5a6466874b20228bb4 WatchSource:0}: Error finding container d13cc3ca48070aa128bf577738f9a9b81c0defe288179c5a6466874b20228bb4: Status 404 returned error can't find the container with id d13cc3ca48070aa128bf577738f9a9b81c0defe288179c5a6466874b20228bb4
Jul 15 05:08:04.044283 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:04.044252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:04.138040 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:04.138011    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:04.260116 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:04.260085    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:04.344751 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:04.344710    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:04.817806 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:04.817745    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr" event={"ID":"1780b906-079c-414c-aa53-aea05c06bbbf","Type":"ContainerStarted","Data":"d13cc3ca48070aa128bf577738f9a9b81c0defe288179c5a6466874b20228bb4"}
Jul 15 05:08:07.832577 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:07.832537    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr" event={"ID":"1780b906-079c-414c-aa53-aea05c06bbbf","Type":"ContainerStarted","Data":"2ef65449344c66dabc795e3fe2aded30e80d20654b503d8d3693537bfa023bd3"}
Jul 15 05:08:08.837933 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:08.837898    2562 generic.go:356] "Generic (PLEG): container finished" podID="1780b906-079c-414c-aa53-aea05c06bbbf" containerID="2ef65449344c66dabc795e3fe2aded30e80d20654b503d8d3693537bfa023bd3" exitCode=0
Jul 15 05:08:08.838405 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:08.837982    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr" event={"ID":"1780b906-079c-414c-aa53-aea05c06bbbf","Type":"ContainerDied","Data":"2ef65449344c66dabc795e3fe2aded30e80d20654b503d8d3693537bfa023bd3"}
Jul 15 05:08:09.844165 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:09.844134    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:09.844584 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:09.844179    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr" event={"ID":"1780b906-079c-414c-aa53-aea05c06bbbf","Type":"ContainerStarted","Data":"e95a1d9991c0ae5ad699a60310cb6d0a2449a8efc76e8751b3b83c705f9a908a"}
Jul 15 05:08:09.864455 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:09.864363    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr" podStartSLOduration=2.375962948 podStartE2EDuration="6.864345784s" totalImagesPullingTime="3.421345661s" totalInitContainerRuntime="1.067037175s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:08:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:08:09.864256689 +0000 UTC m=+572.467809309" watchObservedRunningTime="2026-07-15 05:08:09.864345784 +0000 UTC m=+572.467898393"
Jul 15 05:08:10.803060 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:10.803030    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8fd856859-bmcmr"
Jul 15 05:08:20.851175 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:20.851141    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-749c75689b-sqgqr"
Jul 15 05:08:22.788024 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:22.787989    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:22.885513 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:22.885463    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:24.348171 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.348140    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:24.408120 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.408082    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:24.506362 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.506326    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:24.613969 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.613888    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:24.757843 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.757813    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:24.873684 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.873598    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"]
Jul 15 05:08:24.887157 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.887116    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:24.890384 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.890332    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"]
Jul 15 05:08:24.890525 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.890492    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"
Jul 15 05:08:24.893309 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.893283    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 05:08:24.893477 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.893288    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cc22q\""
Jul 15 05:08:24.893477 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.893317    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 15 05:08:24.893477 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.893341    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 05:08:24.893652 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.893632    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 15 05:08:24.978635 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:24.978603    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:25.007822 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.007783    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61-tmp\") pod \"kube-auth-proxy-57bf8bf567-pzr2h\" (UID: \"0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"
Jul 15 05:08:25.007982 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.007845    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stppc\" (UniqueName: \"kubernetes.io/projected/0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61-kube-api-access-stppc\") pod \"kube-auth-proxy-57bf8bf567-pzr2h\" (UID: \"0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"
Jul 15 05:08:25.007982 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.007902    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61-tls-certs\") pod \"kube-auth-proxy-57bf8bf567-pzr2h\" (UID: \"0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"
Jul 15 05:08:25.085205 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.085174    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:25.108813 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.108764    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stppc\" (UniqueName: \"kubernetes.io/projected/0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61-kube-api-access-stppc\") pod \"kube-auth-proxy-57bf8bf567-pzr2h\" (UID: \"0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"
Jul 15 05:08:25.108813 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.108805    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61-tls-certs\") pod \"kube-auth-proxy-57bf8bf567-pzr2h\" (UID: \"0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"
Jul 15 05:08:25.109037 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.108866    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61-tmp\") pod \"kube-auth-proxy-57bf8bf567-pzr2h\" (UID: \"0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"
Jul 15 05:08:25.111107 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.111075    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61-tmp\") pod \"kube-auth-proxy-57bf8bf567-pzr2h\" (UID: \"0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"
Jul 15 05:08:25.111334 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.111315    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61-tls-certs\") pod \"kube-auth-proxy-57bf8bf567-pzr2h\" (UID: \"0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"
Jul 15 05:08:25.123881 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.123799    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stppc\" (UniqueName: \"kubernetes.io/projected/0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61-kube-api-access-stppc\") pod \"kube-auth-proxy-57bf8bf567-pzr2h\" (UID: \"0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61\") " pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"
Jul 15 05:08:25.175134 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.175101    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:25.200930 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.200885    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"
Jul 15 05:08:25.340233 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.340205    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h"]
Jul 15 05:08:25.342877 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:08:25.342847    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b421add_ecf3_4b58_a6a0_cdfe0fb9cd61.slice/crio-59b5f82d424603a7562e0b98f7c43479a793c4d3dddfb3e78ff103b88682ce3a WatchSource:0}: Error finding container 59b5f82d424603a7562e0b98f7c43479a793c4d3dddfb3e78ff103b88682ce3a: Status 404 returned error can't find the container with id 59b5f82d424603a7562e0b98f7c43479a793c4d3dddfb3e78ff103b88682ce3a
Jul 15 05:08:25.906084 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.906043    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h" event={"ID":"0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61","Type":"ContainerStarted","Data":"59b5f82d424603a7562e0b98f7c43479a793c4d3dddfb3e78ff103b88682ce3a"}
Jul 15 05:08:25.930196 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:25.930161    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:26.028212 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:26.028166    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:26.832379 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:26.832332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:28.919970 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:28.919934    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h" event={"ID":"0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61","Type":"ContainerStarted","Data":"dacc3977f2d7791deefc3816c698f852cecd27cac0c031f99949d9be9ab5a574"}
Jul 15 05:08:28.937459 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:28.937406    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-57bf8bf567-pzr2h" podStartSLOduration=2.188221116 podStartE2EDuration="4.937387987s" totalImagesPullingTime="2.749166871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:08:28.936312614 +0000 UTC m=+591.539865222" watchObservedRunningTime="2026-07-15 05:08:28.937387987 +0000 UTC m=+591.540940586"
Jul 15 05:08:34.445177 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:34.445138    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:34.573295 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:34.573260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:34.916123 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:34.916085    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:35.093569 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:35.093527    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:35.258103 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:35.258077    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:35.439110 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:35.439071    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:35.557219 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:35.557137    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:35.790860 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:35.790827    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:35.892139 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:35.892044    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:35.992667 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:35.992639    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:36.286735 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:36.286703    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:38.153822 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:38.153707    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:08:38.153822 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:38.153707    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:08:40.840883 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:40.840852    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:08:42.382813 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:08:42.382771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:09:24.223708 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:09:24.223597    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:09:24.325554 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:09:24.325497    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:09:24.433991 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:09:24.433957    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:09:25.160742 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:09:25.160705    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:09:25.362208 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:09:25.362174    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:09:50.249674 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:09:50.249634    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:09:51.286100 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:09:51.286066    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:09:54.398047 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:09:54.398019    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:09:58.015571 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:09:58.015534    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:09:58.125307 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:09:58.125272    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:02.433417 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:02.433381    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:02.816385 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:02.816339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:03.837936 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:03.837901    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:06.017701 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:06.017665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:06.309974 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:06.309861    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:08.246267 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.246207    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:08.259090 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.259062    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7bmpd"]
Jul 15 05:10:08.262176 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.262160    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7bmpd"
Jul 15 05:10:08.266401 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.266363    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 05:10:08.266401 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.266392    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 05:10:08.267558 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.267538    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 15 05:10:08.267671 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.267542    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-sz9s2\""
Jul 15 05:10:08.274626 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.274596    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7bmpd"]
Jul 15 05:10:08.408760 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.408728    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtc97\" (UniqueName: \"kubernetes.io/projected/7e748288-9d39-4c24-b5de-717d2362edfc-kube-api-access-rtc97\") pod \"dns-operator-controller-manager-864dd78c4c-7bmpd\" (UID: \"7e748288-9d39-4c24-b5de-717d2362edfc\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7bmpd"
Jul 15 05:10:08.509952 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.509847    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtc97\" (UniqueName: \"kubernetes.io/projected/7e748288-9d39-4c24-b5de-717d2362edfc-kube-api-access-rtc97\") pod \"dns-operator-controller-manager-864dd78c4c-7bmpd\" (UID: \"7e748288-9d39-4c24-b5de-717d2362edfc\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7bmpd"
Jul 15 05:10:08.518757 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.518728    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtc97\" (UniqueName: \"kubernetes.io/projected/7e748288-9d39-4c24-b5de-717d2362edfc-kube-api-access-rtc97\") pod \"dns-operator-controller-manager-864dd78c4c-7bmpd\" (UID: \"7e748288-9d39-4c24-b5de-717d2362edfc\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7bmpd"
Jul 15 05:10:08.572683 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.572647    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7bmpd"
Jul 15 05:10:08.699554 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.699523    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7bmpd"]
Jul 15 05:10:08.702994 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:10:08.702966    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e748288_9d39_4c24_b5de_717d2362edfc.slice/crio-fd57fdf285110fb91da7c4c98ee4ba80031c047c2093f9ec12a0b395d5cd2bb9 WatchSource:0}: Error finding container fd57fdf285110fb91da7c4c98ee4ba80031c047c2093f9ec12a0b395d5cd2bb9: Status 404 returned error can't find the container with id fd57fdf285110fb91da7c4c98ee4ba80031c047c2093f9ec12a0b395d5cd2bb9
Jul 15 05:10:08.705134 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:08.705113    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 05:10:09.306807 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:09.306765    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7bmpd" event={"ID":"7e748288-9d39-4c24-b5de-717d2362edfc","Type":"ContainerStarted","Data":"fd57fdf285110fb91da7c4c98ee4ba80031c047c2093f9ec12a0b395d5cd2bb9"}
Jul 15 05:10:10.043386 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:10.043344    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:10.143223 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:10.143182    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:11.318536 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:11.318496    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7bmpd" event={"ID":"7e748288-9d39-4c24-b5de-717d2362edfc","Type":"ContainerStarted","Data":"a9c6f47d6750910c978485cc15ad11bd00f503dcdbe75410820e8be92ccc7067"}
Jul 15 05:10:11.318942 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:11.318574    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7bmpd"
Jul 15 05:10:11.346874 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:11.346823    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7bmpd" podStartSLOduration=1.296596512 podStartE2EDuration="3.346804486s" totalImagesPullingTime="2.050207974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:10:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:10:11.344878999 +0000 UTC m=+693.948431608" watchObservedRunningTime="2026-07-15 05:10:11.346804486 +0000 UTC m=+693.950357095"
Jul 15 05:10:12.818670 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:12.818631    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:12.823150 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:12.823120    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"]
Jul 15 05:10:12.826320 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:12.826303    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"
Jul 15 05:10:12.828914 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:12.828886    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 15 05:10:12.828914 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:12.828909    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 15 05:10:12.829096 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:12.828993    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-26c86\""
Jul 15 05:10:12.833498 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:12.833476    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"]
Jul 15 05:10:12.949088 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:12.949051    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/922807c7-647b-4b4c-af54-d88b1f65b3f4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-n8q2w\" (UID: \"922807c7-647b-4b4c-af54-d88b1f65b3f4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"
Jul 15 05:10:12.949272 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:12.949105    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/922807c7-647b-4b4c-af54-d88b1f65b3f4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-n8q2w\" (UID: \"922807c7-647b-4b4c-af54-d88b1f65b3f4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"
Jul 15 05:10:12.949272 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:12.949135    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prhxw\" (UniqueName: \"kubernetes.io/projected/922807c7-647b-4b4c-af54-d88b1f65b3f4-kube-api-access-prhxw\") pod \"kuadrant-console-plugin-66b5774985-n8q2w\" (UID: \"922807c7-647b-4b4c-af54-d88b1f65b3f4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"
Jul 15 05:10:12.978330 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:12.978298    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:13.050634 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:13.050597    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prhxw\" (UniqueName: \"kubernetes.io/projected/922807c7-647b-4b4c-af54-d88b1f65b3f4-kube-api-access-prhxw\") pod \"kuadrant-console-plugin-66b5774985-n8q2w\" (UID: \"922807c7-647b-4b4c-af54-d88b1f65b3f4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"
Jul 15 05:10:13.050820 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:13.050664    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/922807c7-647b-4b4c-af54-d88b1f65b3f4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-n8q2w\" (UID: \"922807c7-647b-4b4c-af54-d88b1f65b3f4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"
Jul 15 05:10:13.050820 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:13.050707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/922807c7-647b-4b4c-af54-d88b1f65b3f4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-n8q2w\" (UID: \"922807c7-647b-4b4c-af54-d88b1f65b3f4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"
Jul 15 05:10:13.051475 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:13.051456    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/922807c7-647b-4b4c-af54-d88b1f65b3f4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-n8q2w\" (UID: \"922807c7-647b-4b4c-af54-d88b1f65b3f4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"
Jul 15 05:10:13.053181 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:13.053164    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/922807c7-647b-4b4c-af54-d88b1f65b3f4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-n8q2w\" (UID: \"922807c7-647b-4b4c-af54-d88b1f65b3f4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"
Jul 15 05:10:13.059136 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:13.059109    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prhxw\" (UniqueName: \"kubernetes.io/projected/922807c7-647b-4b4c-af54-d88b1f65b3f4-kube-api-access-prhxw\") pod \"kuadrant-console-plugin-66b5774985-n8q2w\" (UID: \"922807c7-647b-4b4c-af54-d88b1f65b3f4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"
Jul 15 05:10:13.150575 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:13.150481    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"
Jul 15 05:10:13.302687 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:13.302661    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w"]
Jul 15 05:10:13.305366 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:10:13.305336    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod922807c7_647b_4b4c_af54_d88b1f65b3f4.slice/crio-c5d267725ccebfc25da794481297af4f9e8fd58e685a4406cb3d58590234e30c WatchSource:0}: Error finding container c5d267725ccebfc25da794481297af4f9e8fd58e685a4406cb3d58590234e30c: Status 404 returned error can't find the container with id c5d267725ccebfc25da794481297af4f9e8fd58e685a4406cb3d58590234e30c
Jul 15 05:10:13.327089 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:13.327053    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w" event={"ID":"922807c7-647b-4b4c-af54-d88b1f65b3f4","Type":"ContainerStarted","Data":"c5d267725ccebfc25da794481297af4f9e8fd58e685a4406cb3d58590234e30c"}
Jul 15 05:10:13.402493 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:13.402406    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:13.459604 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:13.459572    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:22.324939 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:22.324899    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7bmpd"
Jul 15 05:10:22.473974 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:22.473938    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:22.588888 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:22.588802    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:23.085252 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:23.085215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:23.185610 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:23.185576    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:23.760842 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:23.760626    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k27q4"]
Jul 15 05:10:23.771242 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:23.771211    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k27q4"
Jul 15 05:10:23.775131 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:23.774891    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-qr7fq\""
Jul 15 05:10:23.775750 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:23.775722    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k27q4"]
Jul 15 05:10:23.958048 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:23.958006    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzzn5\" (UniqueName: \"kubernetes.io/projected/3f17e5dd-a08b-41a7-8032-cdec4fcc2ddb-kube-api-access-hzzn5\") pod \"limitador-operator-controller-manager-859d7b78bd-k27q4\" (UID: \"3f17e5dd-a08b-41a7-8032-cdec4fcc2ddb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k27q4"
Jul 15 05:10:24.059539 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:24.059433    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzzn5\" (UniqueName: \"kubernetes.io/projected/3f17e5dd-a08b-41a7-8032-cdec4fcc2ddb-kube-api-access-hzzn5\") pod \"limitador-operator-controller-manager-859d7b78bd-k27q4\" (UID: \"3f17e5dd-a08b-41a7-8032-cdec4fcc2ddb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k27q4"
Jul 15 05:10:24.073156 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:24.073122    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzzn5\" (UniqueName: \"kubernetes.io/projected/3f17e5dd-a08b-41a7-8032-cdec4fcc2ddb-kube-api-access-hzzn5\") pod \"limitador-operator-controller-manager-859d7b78bd-k27q4\" (UID: \"3f17e5dd-a08b-41a7-8032-cdec4fcc2ddb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k27q4"
Jul 15 05:10:24.085923 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:24.085885    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k27q4"
Jul 15 05:10:26.801565 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:26.801499    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:26.861174 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:26.861135    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:29.641026 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:29.640985    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:29.701906 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:29.701866    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:30.383704 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:30.383663    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:30.484330 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:30.484297    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:30.584135 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:30.584103    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:31.164243 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:31.164204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:31.266077 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:31.266042    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:32.195353 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:32.195319    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:32.297764 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:32.297727    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:37.011556 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:37.011522    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:37.143160 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:37.143126    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:37.702822 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:37.702796    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k27q4"]
Jul 15 05:10:37.705655 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:10:37.705629    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f17e5dd_a08b_41a7_8032_cdec4fcc2ddb.slice/crio-284b8cb134125db474bc13f131dcf957293f54b8535409cf11994ea9677999b8 WatchSource:0}: Error finding container 284b8cb134125db474bc13f131dcf957293f54b8535409cf11994ea9677999b8: Status 404 returned error can't find the container with id 284b8cb134125db474bc13f131dcf957293f54b8535409cf11994ea9677999b8
Jul 15 05:10:38.446290 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:38.446195    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w" event={"ID":"922807c7-647b-4b4c-af54-d88b1f65b3f4","Type":"ContainerStarted","Data":"f648da79bb9c8607f197483c0d4cfe3667d0f408dc3eb1aa39d5081610eedd72"}
Jul 15 05:10:38.447583 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:38.447556    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k27q4" event={"ID":"3f17e5dd-a08b-41a7-8032-cdec4fcc2ddb","Type":"ContainerStarted","Data":"284b8cb134125db474bc13f131dcf957293f54b8535409cf11994ea9677999b8"}
Jul 15 05:10:38.462253 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:38.462201    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8q2w" podStartSLOduration=1.763758155 podStartE2EDuration="26.462182373s" totalImagesPullingTime="24.698424218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:10:38.461267197 +0000 UTC m=+721.064819807" watchObservedRunningTime="2026-07-15 05:10:38.462182373 +0000 UTC m=+721.065734982"
Jul 15 05:10:41.246647 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:41.246609    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:41.346761 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:41.346724    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:41.446217 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:41.446172    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:42.469977 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:42.469938    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k27q4" event={"ID":"3f17e5dd-a08b-41a7-8032-cdec4fcc2ddb","Type":"ContainerStarted","Data":"74416bc15931cc8e7e939a2d25a98ca28ed6934281986f303bf7f3144425ac01"}
Jul 15 05:10:42.470495 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:42.470136    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k27q4"
Jul 15 05:10:42.486274 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:42.486226    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k27q4" podStartSLOduration=15.578265935 podStartE2EDuration="19.486208711s" totalImagesPullingTime="3.907942776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:10:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:10:42.485919441 +0000 UTC m=+725.089472053" watchObservedRunningTime="2026-07-15 05:10:42.486208711 +0000 UTC m=+725.089761320"
Jul 15 05:10:46.604061 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:46.604010    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:46.704178 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:46.704146    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:48.599298 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:48.599265    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:49.207522 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:49.207483    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:49.300295 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:49.300261    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:52.407531 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:52.407496    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:52.717156 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:52.717120    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:10:53.476127 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:10:53.476097    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k27q4"
Jul 15 05:11:05.631827 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:05.631794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:05.771144 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:05.771107    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:05.872308 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:05.872274    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:06.571242 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:06.571198    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:06.660350 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:06.660312    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:06.663497 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:06.663466    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-9d9j8"]
Jul 15 05:11:06.904968 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:06.904873    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9d9j8"]
Jul 15 05:11:06.904968 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:06.904936    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9d9j8"
Jul 15 05:11:06.907478 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:06.907448    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2g29f\""
Jul 15 05:11:07.056458 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:07.056413    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4rq6\" (UniqueName: \"kubernetes.io/projected/77e29a72-f792-4637-972d-cb43178d9cc2-kube-api-access-m4rq6\") pod \"authorino-75d556bd77-9d9j8\" (UID: \"77e29a72-f792-4637-972d-cb43178d9cc2\") " pod="kuadrant-system/authorino-75d556bd77-9d9j8"
Jul 15 05:11:07.157843 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:07.157759    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4rq6\" (UniqueName: \"kubernetes.io/projected/77e29a72-f792-4637-972d-cb43178d9cc2-kube-api-access-m4rq6\") pod \"authorino-75d556bd77-9d9j8\" (UID: \"77e29a72-f792-4637-972d-cb43178d9cc2\") " pod="kuadrant-system/authorino-75d556bd77-9d9j8"
Jul 15 05:11:07.166127 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:07.166093    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4rq6\" (UniqueName: \"kubernetes.io/projected/77e29a72-f792-4637-972d-cb43178d9cc2-kube-api-access-m4rq6\") pod \"authorino-75d556bd77-9d9j8\" (UID: \"77e29a72-f792-4637-972d-cb43178d9cc2\") " pod="kuadrant-system/authorino-75d556bd77-9d9j8"
Jul 15 05:11:07.214809 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:07.214767    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9d9j8"
Jul 15 05:11:07.388059 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:07.388027    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9d9j8"]
Jul 15 05:11:07.391114 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:11:07.391087    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77e29a72_f792_4637_972d_cb43178d9cc2.slice/crio-07bde72f5c34d83804cdc0021a61df8e3edfdf795424760629265228e9684c73 WatchSource:0}: Error finding container 07bde72f5c34d83804cdc0021a61df8e3edfdf795424760629265228e9684c73: Status 404 returned error can't find the container with id 07bde72f5c34d83804cdc0021a61df8e3edfdf795424760629265228e9684c73
Jul 15 05:11:07.578258 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:07.578214    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9d9j8" event={"ID":"77e29a72-f792-4637-972d-cb43178d9cc2","Type":"ContainerStarted","Data":"07bde72f5c34d83804cdc0021a61df8e3edfdf795424760629265228e9684c73"}
Jul 15 05:11:10.595670 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:10.595576    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9d9j8" event={"ID":"77e29a72-f792-4637-972d-cb43178d9cc2","Type":"ContainerStarted","Data":"6e111e97e4a5e01dd4d973ae3415539b5377a51305d5c0da1fff1ac6ba9a3e09"}
Jul 15 05:11:10.612565 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:10.612504    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-9d9j8" podStartSLOduration=1.745813104 podStartE2EDuration="4.612486906s" totalImagesPullingTime="2.866673802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:11:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:11:10.610692413 +0000 UTC m=+753.214245022" watchObservedRunningTime="2026-07-15 05:11:10.612486906 +0000 UTC m=+753.216039514"
Jul 15 05:11:20.832635 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:20.832598    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:25.786542 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:25.786491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:26.069295 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:26.069203    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:26.167306 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:26.167267    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:38.717294 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:38.717248    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:38.871151 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:38.871114    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:40.462116 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:40.462061    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9d9j8"]
Jul 15 05:11:40.462582 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:40.462242    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-9d9j8" podUID="77e29a72-f792-4637-972d-cb43178d9cc2" containerName="authorino" containerID="cri-o://6e111e97e4a5e01dd4d973ae3415539b5377a51305d5c0da1fff1ac6ba9a3e09" gracePeriod=30
Jul 15 05:11:40.709701 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:40.709673    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9d9j8"
Jul 15 05:11:40.718478 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:40.718402    2562 generic.go:356] "Generic (PLEG): container finished" podID="77e29a72-f792-4637-972d-cb43178d9cc2" containerID="6e111e97e4a5e01dd4d973ae3415539b5377a51305d5c0da1fff1ac6ba9a3e09" exitCode=0
Jul 15 05:11:40.718606 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:40.718479    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9d9j8" event={"ID":"77e29a72-f792-4637-972d-cb43178d9cc2","Type":"ContainerDied","Data":"6e111e97e4a5e01dd4d973ae3415539b5377a51305d5c0da1fff1ac6ba9a3e09"}
Jul 15 05:11:40.718606 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:40.718488    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9d9j8"
Jul 15 05:11:40.718606 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:40.718509    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9d9j8" event={"ID":"77e29a72-f792-4637-972d-cb43178d9cc2","Type":"ContainerDied","Data":"07bde72f5c34d83804cdc0021a61df8e3edfdf795424760629265228e9684c73"}
Jul 15 05:11:40.718606 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:40.718530    2562 scope.go:117] "RemoveContainer" containerID="6e111e97e4a5e01dd4d973ae3415539b5377a51305d5c0da1fff1ac6ba9a3e09"
Jul 15 05:11:40.726696 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:40.726567    2562 scope.go:117] "RemoveContainer" containerID="6e111e97e4a5e01dd4d973ae3415539b5377a51305d5c0da1fff1ac6ba9a3e09"
Jul 15 05:11:40.726992 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:11:40.726972    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e111e97e4a5e01dd4d973ae3415539b5377a51305d5c0da1fff1ac6ba9a3e09\": container with ID starting with 6e111e97e4a5e01dd4d973ae3415539b5377a51305d5c0da1fff1ac6ba9a3e09 not found: ID does not exist" containerID="6e111e97e4a5e01dd4d973ae3415539b5377a51305d5c0da1fff1ac6ba9a3e09"
Jul 15 05:11:40.727072 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:40.727010    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e111e97e4a5e01dd4d973ae3415539b5377a51305d5c0da1fff1ac6ba9a3e09"} err="failed to get container status \"6e111e97e4a5e01dd4d973ae3415539b5377a51305d5c0da1fff1ac6ba9a3e09\": rpc error: code = NotFound desc = could not find container \"6e111e97e4a5e01dd4d973ae3415539b5377a51305d5c0da1fff1ac6ba9a3e09\": container with ID starting with 6e111e97e4a5e01dd4d973ae3415539b5377a51305d5c0da1fff1ac6ba9a3e09 not found: ID does not exist"
Jul 15 05:11:40.852892 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:40.852862    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4rq6\" (UniqueName: \"kubernetes.io/projected/77e29a72-f792-4637-972d-cb43178d9cc2-kube-api-access-m4rq6\") pod \"77e29a72-f792-4637-972d-cb43178d9cc2\" (UID: \"77e29a72-f792-4637-972d-cb43178d9cc2\") "
Jul 15 05:11:40.855052 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:40.855019    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77e29a72-f792-4637-972d-cb43178d9cc2-kube-api-access-m4rq6" (OuterVolumeSpecName: "kube-api-access-m4rq6") pod "77e29a72-f792-4637-972d-cb43178d9cc2" (UID: "77e29a72-f792-4637-972d-cb43178d9cc2"). InnerVolumeSpecName "kube-api-access-m4rq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:11:40.953730 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:40.953693    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4rq6\" (UniqueName: \"kubernetes.io/projected/77e29a72-f792-4637-972d-cb43178d9cc2-kube-api-access-m4rq6\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:11:41.039643 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:41.039608    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9d9j8"]
Jul 15 05:11:41.043456 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:41.043433    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9d9j8"]
Jul 15 05:11:41.850020 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:41.849725    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77e29a72-f792-4637-972d-cb43178d9cc2" path="/var/lib/kubelet/pods/77e29a72-f792-4637-972d-cb43178d9cc2/volumes"
Jul 15 05:11:45.165251 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:45.165215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:45.264873 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:45.264837    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:45.323428 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:45.323392    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:45.610835 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:45.610800    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:48.282212 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:48.282175    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:11:48.382726 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:11:48.382688    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:12:06.819294 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:06.819210    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:12:06.920185 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:06.920142    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:12:20.092695 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:20.092662    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:12:20.195306 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:20.195270    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:12:20.593622 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:20.593583    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:12:21.044560 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:21.044527    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:12:21.141288 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:21.141236    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:12:21.321772 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:21.321678    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:12:31.664995 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:31.664953    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:12:31.765797 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:31.765761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:12:47.935781 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:47.935748    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"]
Jul 15 05:12:47.936159 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:47.936047    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77e29a72-f792-4637-972d-cb43178d9cc2" containerName="authorino"
Jul 15 05:12:47.936159 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:47.936057    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="77e29a72-f792-4637-972d-cb43178d9cc2" containerName="authorino"
Jul 15 05:12:47.936159 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:47.936116    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="77e29a72-f792-4637-972d-cb43178d9cc2" containerName="authorino"
Jul 15 05:12:47.939638 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:47.939621    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:47.943758 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:47.943728    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 15 05:12:47.943905 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:47.943774    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 05:12:47.943905 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:47.943790    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zcd2w\""
Jul 15 05:12:47.943905 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:47.943731    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 05:12:47.950731 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:47.950708    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"]
Jul 15 05:12:48.033005 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.032965    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fda1dee7-5826-4fa5-96be-e25fc312e74a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.033005 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.033010    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fda1dee7-5826-4fa5-96be-e25fc312e74a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.033212 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.033068    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda1dee7-5826-4fa5-96be-e25fc312e74a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.033212 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.033128    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fda1dee7-5826-4fa5-96be-e25fc312e74a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.033212 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.033187    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nmn8\" (UniqueName: \"kubernetes.io/projected/fda1dee7-5826-4fa5-96be-e25fc312e74a-kube-api-access-6nmn8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.033212 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.033210    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fda1dee7-5826-4fa5-96be-e25fc312e74a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.041282 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.041250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:12:48.134580 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.134541    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fda1dee7-5826-4fa5-96be-e25fc312e74a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.134759 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.134589    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fda1dee7-5826-4fa5-96be-e25fc312e74a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.134759 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.134623    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda1dee7-5826-4fa5-96be-e25fc312e74a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.134759 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.134666    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fda1dee7-5826-4fa5-96be-e25fc312e74a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.134759 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.134728    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nmn8\" (UniqueName: \"kubernetes.io/projected/fda1dee7-5826-4fa5-96be-e25fc312e74a-kube-api-access-6nmn8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.134759 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.134758    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fda1dee7-5826-4fa5-96be-e25fc312e74a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.135025 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.134976    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fda1dee7-5826-4fa5-96be-e25fc312e74a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.135025 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.135015    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fda1dee7-5826-4fa5-96be-e25fc312e74a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.135132 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.135096    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fda1dee7-5826-4fa5-96be-e25fc312e74a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.136942 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.136921    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fda1dee7-5826-4fa5-96be-e25fc312e74a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.137099 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.137084    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fda1dee7-5826-4fa5-96be-e25fc312e74a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.142446 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.142423    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nmn8\" (UniqueName: \"kubernetes.io/projected/fda1dee7-5826-4fa5-96be-e25fc312e74a-kube-api-access-6nmn8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq\" (UID: \"fda1dee7-5826-4fa5-96be-e25fc312e74a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.250252 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.250194    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:48.383546 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:48.383520    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"]
Jul 15 05:12:48.385138 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:12:48.385110    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfda1dee7_5826_4fa5_96be_e25fc312e74a.slice/crio-d3052a954a359dea7661352db8b337e32ef05bb821377060cebf4dcacd4387f6 WatchSource:0}: Error finding container d3052a954a359dea7661352db8b337e32ef05bb821377060cebf4dcacd4387f6: Status 404 returned error can't find the container with id d3052a954a359dea7661352db8b337e32ef05bb821377060cebf4dcacd4387f6
Jul 15 05:12:49.006974 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:49.006922    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq" event={"ID":"fda1dee7-5826-4fa5-96be-e25fc312e74a","Type":"ContainerStarted","Data":"d3052a954a359dea7661352db8b337e32ef05bb821377060cebf4dcacd4387f6"}
Jul 15 05:12:50.012179 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:50.012141    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq" event={"ID":"fda1dee7-5826-4fa5-96be-e25fc312e74a","Type":"ContainerStarted","Data":"8b0ff049d08763f7d0cf767ae8b1b95b9a13c127af21e24daba4aef01afe5190"}
Jul 15 05:12:50.032464 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:50.032402    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq" podStartSLOduration=1.971584405 podStartE2EDuration="3.03238679s" totalImagesPullingTime="1.060802385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:12:50.03025509 +0000 UTC m=+852.633807700" watchObservedRunningTime="2026-07-15 05:12:50.03238679 +0000 UTC m=+852.635939390"
Jul 15 05:12:58.251335 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:58.251284    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:58.251975 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:58.251361    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:58.264106 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:58.264076    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:12:59.066067 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:12:59.066037    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq"
Jul 15 05:13:00.616473 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:00.616430    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:13:16.963553 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:16.963513    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:13:27.436184 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.436143    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"]
Jul 15 05:13:27.442634 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.442609    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.445054 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.445030    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 15 05:13:27.448050 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.448028    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"]
Jul 15 05:13:27.466408 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.466379    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:13:27.577432 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.577365    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c88f43e2-8be8-401c-a146-ccf40c1b06ed-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.577623 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.577446    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c88f43e2-8be8-401c-a146-ccf40c1b06ed-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.577623 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.577474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c88f43e2-8be8-401c-a146-ccf40c1b06ed-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.577623 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.577489    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c88f43e2-8be8-401c-a146-ccf40c1b06ed-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.577623 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.577506    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c88f43e2-8be8-401c-a146-ccf40c1b06ed-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.577623 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.577599    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-528c4\" (UniqueName: \"kubernetes.io/projected/c88f43e2-8be8-401c-a146-ccf40c1b06ed-kube-api-access-528c4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.678603 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.678569    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c88f43e2-8be8-401c-a146-ccf40c1b06ed-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.678792 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.678620    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c88f43e2-8be8-401c-a146-ccf40c1b06ed-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.678792 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.678656    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c88f43e2-8be8-401c-a146-ccf40c1b06ed-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.678792 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.678678    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c88f43e2-8be8-401c-a146-ccf40c1b06ed-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.678792 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.678704    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c88f43e2-8be8-401c-a146-ccf40c1b06ed-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.678792 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.678735    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-528c4\" (UniqueName: \"kubernetes.io/projected/c88f43e2-8be8-401c-a146-ccf40c1b06ed-kube-api-access-528c4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.679147 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.679112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c88f43e2-8be8-401c-a146-ccf40c1b06ed-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.679252 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.679231    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c88f43e2-8be8-401c-a146-ccf40c1b06ed-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.679309 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.679284    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c88f43e2-8be8-401c-a146-ccf40c1b06ed-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.680970 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.680947    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c88f43e2-8be8-401c-a146-ccf40c1b06ed-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.681253 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.681233    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c88f43e2-8be8-401c-a146-ccf40c1b06ed-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.686777 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.686719    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-528c4\" (UniqueName: \"kubernetes.io/projected/c88f43e2-8be8-401c-a146-ccf40c1b06ed-kube-api-access-528c4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lsk7h\" (UID: \"c88f43e2-8be8-401c-a146-ccf40c1b06ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.778035 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.777985    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:27.909942 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:27.909914    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"]
Jul 15 05:13:27.911895 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:13:27.911869    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc88f43e2_8be8_401c_a146_ccf40c1b06ed.slice/crio-88e3473d590e46232b4f1641e432282f2cf26d6c1ec98d9d6d793ed9fbbd6e74 WatchSource:0}: Error finding container 88e3473d590e46232b4f1641e432282f2cf26d6c1ec98d9d6d793ed9fbbd6e74: Status 404 returned error can't find the container with id 88e3473d590e46232b4f1641e432282f2cf26d6c1ec98d9d6d793ed9fbbd6e74
Jul 15 05:13:28.179074 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:28.179033    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h" event={"ID":"c88f43e2-8be8-401c-a146-ccf40c1b06ed","Type":"ContainerStarted","Data":"a99ad8c6538e24d37601f6ed21b84a0db1d118817086652e8eec10b5723dcdba"}
Jul 15 05:13:28.179254 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:28.179140    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h" event={"ID":"c88f43e2-8be8-401c-a146-ccf40c1b06ed","Type":"ContainerStarted","Data":"88e3473d590e46232b4f1641e432282f2cf26d6c1ec98d9d6d793ed9fbbd6e74"}
Jul 15 05:13:28.196676 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:28.196628    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h" podStartSLOduration=1.014183298 podStartE2EDuration="1.19661129s" totalImagesPullingTime="182.427992ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:13:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:13:28.195499816 +0000 UTC m=+890.799052425" watchObservedRunningTime="2026-07-15 05:13:28.19661129 +0000 UTC m=+890.800163899"
Jul 15 05:13:37.534537 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:37.534492    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:13:37.778208 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:37.778174    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:37.778421 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:37.778240    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:37.790652 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:37.790579    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:38.180878 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:38.180751    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:13:38.185062 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:38.180925    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:13:38.232920 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:38.232886    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lsk7h"
Jul 15 05:13:39.465872 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:39.465825    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:13:47.727315 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.727280    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"]
Jul 15 05:13:47.730615 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.730598    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.733486 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.733468    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 15 05:13:47.740558 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.740538    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"]
Jul 15 05:13:47.754626 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.754594    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2099183-1f82-4dc7-88de-519cf3097d75-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.754826 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.754797    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2099183-1f82-4dc7-88de-519cf3097d75-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.754919 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.754870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2099183-1f82-4dc7-88de-519cf3097d75-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.754919 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.754910    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2099183-1f82-4dc7-88de-519cf3097d75-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.755028 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.754943    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2099183-1f82-4dc7-88de-519cf3097d75-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.755028 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.754996    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkmvq\" (UniqueName: \"kubernetes.io/projected/c2099183-1f82-4dc7-88de-519cf3097d75-kube-api-access-dkmvq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.856382 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.856350    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2099183-1f82-4dc7-88de-519cf3097d75-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.856571 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.856404    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2099183-1f82-4dc7-88de-519cf3097d75-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.856571 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.856436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2099183-1f82-4dc7-88de-519cf3097d75-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.856571 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.856462    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2099183-1f82-4dc7-88de-519cf3097d75-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.856571 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.856482    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2099183-1f82-4dc7-88de-519cf3097d75-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.856571 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.856499    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkmvq\" (UniqueName: \"kubernetes.io/projected/c2099183-1f82-4dc7-88de-519cf3097d75-kube-api-access-dkmvq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.856838 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.856789    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2099183-1f82-4dc7-88de-519cf3097d75-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.856838 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.856806    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2099183-1f82-4dc7-88de-519cf3097d75-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.856936 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.856863    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2099183-1f82-4dc7-88de-519cf3097d75-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.858788 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.858760    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2099183-1f82-4dc7-88de-519cf3097d75-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.859098 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.859080    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2099183-1f82-4dc7-88de-519cf3097d75-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:47.865033 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:47.865015    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkmvq\" (UniqueName: \"kubernetes.io/projected/c2099183-1f82-4dc7-88de-519cf3097d75-kube-api-access-dkmvq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z\" (UID: \"c2099183-1f82-4dc7-88de-519cf3097d75\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:48.040469 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:48.040348    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:48.173407 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:48.173362    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"]
Jul 15 05:13:48.175431 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:13:48.175403    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2099183_1f82_4dc7_88de_519cf3097d75.slice/crio-c1c31541e9f633fa542db3509656c5536f5c3282e7b0184f181c20adf2660b9b WatchSource:0}: Error finding container c1c31541e9f633fa542db3509656c5536f5c3282e7b0184f181c20adf2660b9b: Status 404 returned error can't find the container with id c1c31541e9f633fa542db3509656c5536f5c3282e7b0184f181c20adf2660b9b
Jul 15 05:13:48.263672 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:48.263636    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z" event={"ID":"c2099183-1f82-4dc7-88de-519cf3097d75","Type":"ContainerStarted","Data":"c1c31541e9f633fa542db3509656c5536f5c3282e7b0184f181c20adf2660b9b"}
Jul 15 05:13:49.268998 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:49.268962    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z" event={"ID":"c2099183-1f82-4dc7-88de-519cf3097d75","Type":"ContainerStarted","Data":"ff7780dcbde0563199b880941eac5b951cf4a7f0815b5f19c187c6588d9bf8ad"}
Jul 15 05:13:49.289322 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:49.289257    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z" podStartSLOduration=2.081825511 podStartE2EDuration="2.289238439s" totalImagesPullingTime="207.412928ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:13:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:13:49.287022736 +0000 UTC m=+911.890575346" watchObservedRunningTime="2026-07-15 05:13:49.289238439 +0000 UTC m=+911.892791051"
Jul 15 05:13:58.040494 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:58.040454    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:58.040938 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:58.040511    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:58.053168 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:58.053139    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:13:58.319556 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:13:58.319462    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z"
Jul 15 05:14:44.244074 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:14:44.244034    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:14:46.505181 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:14:46.505144    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:14:46.584042 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:14:46.584010    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:14:47.667411 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:14:47.667357    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:14:47.764231 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:14:47.764189    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:15:00.756209 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:15:00.756135    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:15:36.351965 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:15:36.351919    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:15:36.452801 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:15:36.452761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:16:10.163313 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:16:10.163267    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:16:13.874775 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:16:13.874737    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:16:15.503948 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:16:15.503916    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:16:58.677581 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:16:58.677534    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:16:58.772910 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:16:58.772868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:16:58.901396 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:16:58.901334    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:16:59.431345 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:16:59.431307    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:16:59.487997 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:16:59.487963    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:17:04.699226 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:17:04.699187    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:17:04.755738 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:17:04.755705    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:17:04.856830 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:17:04.856789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:17:04.953313 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:17:04.953228    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:17:05.128663 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:17:05.128634    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:17:10.898918 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:17:10.898881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:17:10.963198 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:17:10.963166    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:18:38.206337 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:18:38.206310    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:18:38.208561 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:18:38.207515    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:21:14.146621 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:21:14.146530    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:23:31.132146 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:23:31.132114    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:23:38.233132 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:23:38.233021    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:23:38.236199 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:23:38.234742    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:25:33.964126 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:25:33.964083    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:25:34.055147 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:25:34.055106    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:15.439614 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:15.439580    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:22.923397 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:22.923332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:31.202817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:31.202723    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:36.305506 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:36.305470    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:38.258717 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:38.258602    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:28:38.265708 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:38.262043    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:28:39.474864 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:39.474830    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:39.569961 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:39.569928    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:39.937985 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:39.937905    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:45.665544 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:45.665505    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:45.728654 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:45.728614    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:45.898612 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:45.898583    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:45.992803 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:45.992753    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:47.089799 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:47.089765    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:47.169413 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:47.169321    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:47.242158 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:47.242120    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:47.342201 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:47.342118    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:47.826520 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:47.826485    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:47.929643 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:47.929609    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:28:50.064411 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:28:50.064347    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:04.127192 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:04.127154    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:04.187872 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:04.187831    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:12.591859 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:12.591829    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:17.223939 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:17.223904    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:22.975980 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:22.975946    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:23.076042 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:23.076011    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:27.431725 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:27.431678    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:27.783656 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:27.783615    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:27.884868 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:27.884839    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:27.986986 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:27.986955    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:29.199810 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:29.199775    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:29.302989 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:29.302960    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:29.571112 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:29.571013    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:29.787122 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:29.787075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:29.880974 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:29.880884    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:34.105924 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:34.105893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:34.164320 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:34.164286    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:29:37.648091 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:29:37.648060    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:33.922214 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:33.922128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:34.025631 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:34.025592    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:34.294981 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:34.294952    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:39.793673 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:39.793637    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:39.861360 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:39.861328    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:40.025575 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:40.025543    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:41.151136 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:41.151100    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:41.254438 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:41.254401    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:41.396547 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:41.396514    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:41.802064 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:41.802026    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:41.935941 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:41.935901    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:42.038727 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:42.038694    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:45.194897 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:45.194861    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:45.250609 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:45.250568    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:30:48.435553 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:30:48.435515    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:31:37.920694 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:31:37.920608    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:31:38.020203 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:31:38.020169    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:31:47.063399 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:31:47.063338    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:31:47.167775 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:31:47.167740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:31:47.321073 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:31:47.320960    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:31:51.448575 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:31:51.448533    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:31:57.447221 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:31:57.447143    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:31:57.508448 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:31:57.508407    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:32:02.727833 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:32:02.727775    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:32:03.151851 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:32:03.151820    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:32:04.330061 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:32:04.330030    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:32:04.474021 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:32:04.473990    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:32:04.573986 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:32:04.573951    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:32:04.672093 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:32:04.672015    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:32:05.103325 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:32:05.103258    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:32:05.214279 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:32:05.214245    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:32:09.304717 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:32:09.304671    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:32:09.404150 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:32:09.404114    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:32:12.965486 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:32:12.965439    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:19.215180 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:19.215095    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:23.905590 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:23.905556    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:29.557859 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:29.557823    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:29.657743 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:29.657708    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:34.309993 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:34.309958    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:35.404566 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:35.404528    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:35.504203 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:35.504169    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:35.604570 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:35.604535    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:38.285119 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:38.284997    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:33:38.290291 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:38.290248    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:33:39.040679 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:39.040639    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:44.652836 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:44.652795    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:44.707178 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:44.707138    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:44.801235 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:44.801200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:44.898217 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:44.898171    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:44.998534 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:44.998500    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:45.967455 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:45.967413    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:33:46.070034 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:33:46.070005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:12.039665 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:12.039628    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:17.601001 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:17.600966    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:17.704040 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:17.704003    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:17.800825 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:17.800790    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:23.091819 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:23.091780    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:23.143304 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:23.143265    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:34.765519 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:34.765439    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:40.522961 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:40.522926    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:40.623818 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:40.623783    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:40.727359 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:40.727319    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:40.825238 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:40.825157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:51.604806 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:51.604771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:52.409973 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:52.409941    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:58.194304 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:58.194273    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:58.294538 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:58.294504    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:58.455856 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:58.455822    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:58.574936 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:58.574896    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:59.435187 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:59.435156    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:59.535712 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:59.535674    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:34:59.635781 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:34:59.635734    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:01.676775 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:01.676744    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:05.960761 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:05.960717    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:07.350294 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:07.350260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:07.406806 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:07.406775    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:08.038534 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:08.038502    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:13.570572 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:13.570538    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:13.671894 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:13.671862    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:13.820138 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:13.820104    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:14.126523 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:14.126471    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:14.489235 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:14.489191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:16.818219 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:16.818188    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:16.922230 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:16.922195    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:16.985883 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:16.985848    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:17.237828 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:17.237795    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:17.658656 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:17.658558    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:17.760925 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:17.760891    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:21.305308 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:21.305278    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:21.945960 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:21.945924    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:22.044781 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:22.044744    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:22.393906 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:22.393873    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:27.998934 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:27.998894    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:28.058580 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:28.058539    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:38.759998 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:38.759943    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:43.443461 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:43.443426    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:49.190860 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:49.190826    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:49.289931 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:49.289897    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:54.154227 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:54.154189    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:54.391024 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:54.390995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:54.515664 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:54.515636    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:54.617126 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:54.617094    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:35:54.753820 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:35:54.753783    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:00.391948 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:00.391853    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:00.455663 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:00.455619    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:00.550026 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:00.549991    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:00.645683 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:00.645565    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:00.802513 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:00.802485    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:06.308090 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:06.308054    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:06.365885 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:06.365847    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:15.447638 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:15.447600    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:15.782261 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:15.782149    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:16.073086 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:16.072989    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:19.100380 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:19.100325    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:19.192388 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:19.192283    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:19.263939 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:19.263897    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:19.360815 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:19.360727    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:19.748105 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:19.748070    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:19.849388 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:19.849330    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:23.738265 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:23.738212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:31.531715 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.531675    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"]
Jul 15 05:36:31.535407 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.535382    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.537828 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.537769    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f84d43c4\"/\"default-dockercfg-9b9cs\""
Jul 15 05:36:31.537958 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.537918    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f84d43c4\"/\"kube-root-ca.crt\""
Jul 15 05:36:31.538892 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.538872    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f84d43c4\"/\"openshift-service-ca.crt\""
Jul 15 05:36:31.539010 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.538901    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f84d43c4\"/\"e2e-lifecycle-model-f84d43c4-kserve-self-signed-certs\""
Jul 15 05:36:31.541987 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.541962    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"]
Jul 15 05:36:31.560724 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.560692    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:31.667121 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.667088    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-model-cache\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.667289 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.667132    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aa6e7a4-24d4-47f3-b936-83cca0195032-tls-certs\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.667289 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.667172    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8w5t\" (UniqueName: \"kubernetes.io/projected/3aa6e7a4-24d4-47f3-b936-83cca0195032-kube-api-access-j8w5t\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.667289 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.667195    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-home\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.667289 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.667220    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-tmp-dir\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.667289 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.667245    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-dshm\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.768458 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.768421    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-model-cache\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.768636 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.768478    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aa6e7a4-24d4-47f3-b936-83cca0195032-tls-certs\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.768636 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.768518    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8w5t\" (UniqueName: \"kubernetes.io/projected/3aa6e7a4-24d4-47f3-b936-83cca0195032-kube-api-access-j8w5t\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.768636 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.768541    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-home\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.768636 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.768568    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-tmp-dir\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.768636 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.768612    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-dshm\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.768893 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.768870    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-model-cache\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.768956 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.768891    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-tmp-dir\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.769012 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.768948    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-home\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.770764 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.770739    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-dshm\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.771103 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.771086    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aa6e7a4-24d4-47f3-b936-83cca0195032-tls-certs\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.776081 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.776063    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8w5t\" (UniqueName: \"kubernetes.io/projected/3aa6e7a4-24d4-47f3-b936-83cca0195032-kube-api-access-j8w5t\") pod \"e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") " pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.846609 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.846529    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:31.978299 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:31.978273    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"]
Jul 15 05:36:31.981114 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:36:31.981091    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3aa6e7a4_24d4_47f3_b936_83cca0195032.slice/crio-9f293022747574335a9a0b0930371f50882bc9f33321b1af909cb528144131b4 WatchSource:0}: Error finding container 9f293022747574335a9a0b0930371f50882bc9f33321b1af909cb528144131b4: Status 404 returned error can't find the container with id 9f293022747574335a9a0b0930371f50882bc9f33321b1af909cb528144131b4
Jul 15 05:36:32.209432 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:32.209365    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk" event={"ID":"3aa6e7a4-24d4-47f3-b936-83cca0195032","Type":"ContainerStarted","Data":"5e0d8d2a8a5c4f741c7a59598922ce1ecbb5bbebb54c20db77c6695765475676"}
Jul 15 05:36:32.209624 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:32.209441    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk" event={"ID":"3aa6e7a4-24d4-47f3-b936-83cca0195032","Type":"ContainerStarted","Data":"9f293022747574335a9a0b0930371f50882bc9f33321b1af909cb528144131b4"}
Jul 15 05:36:32.224979 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:32.224916    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk" podStartSLOduration=1.224898365 podStartE2EDuration="1.224898365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:36:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:36:32.224193477 +0000 UTC m=+2274.827746088" watchObservedRunningTime="2026-07-15 05:36:32.224898365 +0000 UTC m=+2274.828450975"
Jul 15 05:36:38.617258 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:38.617225    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:38.717742 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:38.717701    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:40.461248 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:40.461209    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:40.563597 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:40.563563    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:41.846909 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:41.846876    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:41.847286 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:41.846929    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:41.859930 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:41.859899    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:42.263653 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:42.263620    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:36:51.415637 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:51.415603    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:36:51.479656 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:36:51.479626    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:37:12.593177 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:12.593146    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:37:17.250846 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:17.250809    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:37:22.232750 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:22.232716    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"]
Jul 15 05:37:22.329193 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:22.329158    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:37:37.253599 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.253507    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk" podUID="3aa6e7a4-24d4-47f3-b936-83cca0195032" containerName="main" containerID="cri-o://5e0d8d2a8a5c4f741c7a59598922ce1ecbb5bbebb54c20db77c6695765475676" gracePeriod=45
Jul 15 05:37:37.498486 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.498454    2562 generic.go:356] "Generic (PLEG): container finished" podID="3aa6e7a4-24d4-47f3-b936-83cca0195032" containerID="5e0d8d2a8a5c4f741c7a59598922ce1ecbb5bbebb54c20db77c6695765475676" exitCode=0
Jul 15 05:37:37.498633 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.498527    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk" event={"ID":"3aa6e7a4-24d4-47f3-b936-83cca0195032","Type":"ContainerDied","Data":"5e0d8d2a8a5c4f741c7a59598922ce1ecbb5bbebb54c20db77c6695765475676"}
Jul 15 05:37:37.498633 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.498567    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk" event={"ID":"3aa6e7a4-24d4-47f3-b936-83cca0195032","Type":"ContainerDied","Data":"9f293022747574335a9a0b0930371f50882bc9f33321b1af909cb528144131b4"}
Jul 15 05:37:37.498633 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.498579    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f293022747574335a9a0b0930371f50882bc9f33321b1af909cb528144131b4"
Jul 15 05:37:37.504701 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.504647    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:37:37.584916 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.584880    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-tmp-dir\") pod \"3aa6e7a4-24d4-47f3-b936-83cca0195032\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") "
Jul 15 05:37:37.584916 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.584917    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-dshm\") pod \"3aa6e7a4-24d4-47f3-b936-83cca0195032\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") "
Jul 15 05:37:37.585160 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.584942    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-home\") pod \"3aa6e7a4-24d4-47f3-b936-83cca0195032\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") "
Jul 15 05:37:37.585160 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.585021    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aa6e7a4-24d4-47f3-b936-83cca0195032-tls-certs\") pod \"3aa6e7a4-24d4-47f3-b936-83cca0195032\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") "
Jul 15 05:37:37.585160 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.585045    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-model-cache\") pod \"3aa6e7a4-24d4-47f3-b936-83cca0195032\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") "
Jul 15 05:37:37.585160 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.585065    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8w5t\" (UniqueName: \"kubernetes.io/projected/3aa6e7a4-24d4-47f3-b936-83cca0195032-kube-api-access-j8w5t\") pod \"3aa6e7a4-24d4-47f3-b936-83cca0195032\" (UID: \"3aa6e7a4-24d4-47f3-b936-83cca0195032\") "
Jul 15 05:37:37.585389 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.585205    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3aa6e7a4-24d4-47f3-b936-83cca0195032" (UID: "3aa6e7a4-24d4-47f3-b936-83cca0195032"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:37:37.585389 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.585234    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-model-cache" (OuterVolumeSpecName: "model-cache") pod "3aa6e7a4-24d4-47f3-b936-83cca0195032" (UID: "3aa6e7a4-24d4-47f3-b936-83cca0195032"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:37:37.585389 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.585252    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-home" (OuterVolumeSpecName: "home") pod "3aa6e7a4-24d4-47f3-b936-83cca0195032" (UID: "3aa6e7a4-24d4-47f3-b936-83cca0195032"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:37:37.585389 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.585347    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-model-cache\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:37:37.585389 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.585361    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-tmp-dir\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:37:37.585562 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.585394    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-home\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:37:37.587187 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.587163    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3aa6e7a4-24d4-47f3-b936-83cca0195032-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3aa6e7a4-24d4-47f3-b936-83cca0195032" (UID: "3aa6e7a4-24d4-47f3-b936-83cca0195032"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:37:37.587187 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.587177    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aa6e7a4-24d4-47f3-b936-83cca0195032-kube-api-access-j8w5t" (OuterVolumeSpecName: "kube-api-access-j8w5t") pod "3aa6e7a4-24d4-47f3-b936-83cca0195032" (UID: "3aa6e7a4-24d4-47f3-b936-83cca0195032"). InnerVolumeSpecName "kube-api-access-j8w5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:37:37.587340 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.587171    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-dshm" (OuterVolumeSpecName: "dshm") pod "3aa6e7a4-24d4-47f3-b936-83cca0195032" (UID: "3aa6e7a4-24d4-47f3-b936-83cca0195032"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:37:37.686730 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.686688    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aa6e7a4-24d4-47f3-b936-83cca0195032-dshm\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:37:37.686730 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.686714    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aa6e7a4-24d4-47f3-b936-83cca0195032-tls-certs\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:37:37.686730 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:37.686725    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8w5t\" (UniqueName: \"kubernetes.io/projected/3aa6e7a4-24d4-47f3-b936-83cca0195032-kube-api-access-j8w5t\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:37:38.503246 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:38.503217    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"
Jul 15 05:37:38.518662 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:38.518628    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"]
Jul 15 05:37:38.523611 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:38.523586    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-f84d43c4/e2e-lifecycle-model-f84d43c4-kserve-6d565dcfff-r82pk"]
Jul 15 05:37:39.844502 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:39.844464    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3aa6e7a4-24d4-47f3-b936-83cca0195032" path="/var/lib/kubelet/pods/3aa6e7a4-24d4-47f3-b936-83cca0195032/volumes"
Jul 15 05:37:59.104071 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:59.104038    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:37:59.254194 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:37:59.254157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:04.713207 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:04.713170    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:10.150347 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:10.150310    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:10.202238 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:10.202204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:19.623363 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:19.623332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:19.847695 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:19.847662    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:20.889659 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:20.889620    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:22.103881 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:22.103843    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:23.456515 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:23.456482    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:29.388781 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:29.388742    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:29.444956 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:29.444920    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:30.113273 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:30.113239    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:35.725796 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:35.725761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:35.832216 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:35.832184    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:35.928172 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:35.928138    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:38.311122 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:38.310988    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:38:38.317530 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:38.317509    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:38:46.460408 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:46.460359    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:38:46.815026 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:38:46.814936    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:07.847805 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:07.847714    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:13.394601 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:13.394569    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:13.494654 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:13.494618    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:13.730338 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:13.730304    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:14.062140 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:14.062056    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:14.179227 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:14.179188    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:16.994670 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:16.994635    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:17.089750 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:17.089712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:17.160774 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:17.160740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:17.533140 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:17.533104    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:17.633467 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:17.633433    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:17.734392 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:17.734339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:21.520970 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:21.520936    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:21.864757 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:21.864672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:22.181137 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:22.181057    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:22.360113 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:22.360080    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:25.042458 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:25.042423    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:25.132032 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:25.132002    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:25.198311 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:25.198276    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:25.412505 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:25.412418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:25.673526 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:25.673445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:25.786038 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:25.786002    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:29.557206 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:29.557177    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:34.747154 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.747120    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"]
Jul 15 05:39:34.747550 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.747494    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aa6e7a4-24d4-47f3-b936-83cca0195032" containerName="main"
Jul 15 05:39:34.747550 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.747507    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aa6e7a4-24d4-47f3-b936-83cca0195032" containerName="main"
Jul 15 05:39:34.747621 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.747574    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aa6e7a4-24d4-47f3-b936-83cca0195032" containerName="main"
Jul 15 05:39:34.750689 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.750671    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.754101 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.754073    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-891b7e\"/\"kube-root-ca.crt\""
Jul 15 05:39:34.754101 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.754094    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-891b7e\"/\"openshift-service-ca.crt\""
Jul 15 05:39:34.754317 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.754078    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-891b7e\"/\"default-dockercfg-x2hbg\""
Jul 15 05:39:34.754317 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.754119    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-891b7e\"/\"test-model-891b7e-kserve-self-signed-certs\""
Jul 15 05:39:34.757856 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.757824    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"]
Jul 15 05:39:34.770616 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.770586    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:34.858110 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.858070    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039e0b63-fbdb-425f-985f-6d6b1f1adc52-tls-certs\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.858295 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.858123    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-home\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.858295 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.858209    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-tmp-dir\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.858295 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.858247    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dpdh\" (UniqueName: \"kubernetes.io/projected/039e0b63-fbdb-425f-985f-6d6b1f1adc52-kube-api-access-8dpdh\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.858477 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.858294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-model-cache\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.858477 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.858414    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-dshm\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.959359 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.959316    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-home\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.959553 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.959398    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-tmp-dir\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.959553 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.959420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dpdh\" (UniqueName: \"kubernetes.io/projected/039e0b63-fbdb-425f-985f-6d6b1f1adc52-kube-api-access-8dpdh\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.959553 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.959451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-model-cache\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.959553 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.959468    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-dshm\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.959553 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.959499    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039e0b63-fbdb-425f-985f-6d6b1f1adc52-tls-certs\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.959819 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.959745    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-home\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.959819 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.959794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-tmp-dir\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.959899 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.959834    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-model-cache\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.961733 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.961710    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-dshm\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.962045 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.962029    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039e0b63-fbdb-425f-985f-6d6b1f1adc52-tls-certs\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:34.967751 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:34.967732    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dpdh\" (UniqueName: \"kubernetes.io/projected/039e0b63-fbdb-425f-985f-6d6b1f1adc52-kube-api-access-8dpdh\") pod \"test-model-891b7e-kserve-5f565cd585-vwkh8\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") " pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:35.061442 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:35.061323    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:35.204866 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:35.204841    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"]
Jul 15 05:39:35.207608 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:39:35.207577    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod039e0b63_fbdb_425f_985f_6d6b1f1adc52.slice/crio-703c7c4a768c5eafe57e33010715c0266d7757ff7fc64999c4366083ba3f455e WatchSource:0}: Error finding container 703c7c4a768c5eafe57e33010715c0266d7757ff7fc64999c4366083ba3f455e: Status 404 returned error can't find the container with id 703c7c4a768c5eafe57e33010715c0266d7757ff7fc64999c4366083ba3f455e
Jul 15 05:39:36.009868 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:36.009834    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8" event={"ID":"039e0b63-fbdb-425f-985f-6d6b1f1adc52","Type":"ContainerStarted","Data":"6dcf3d74139733c8993d81e26f998e4f8921c5bce5cdcb1a70f1cd3dbc1cdcde"}
Jul 15 05:39:36.010266 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:36.009874    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8" event={"ID":"039e0b63-fbdb-425f-985f-6d6b1f1adc52","Type":"ContainerStarted","Data":"703c7c4a768c5eafe57e33010715c0266d7757ff7fc64999c4366083ba3f455e"}
Jul 15 05:39:36.026719 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:36.026664    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8" podStartSLOduration=2.026647833 podStartE2EDuration="2.026647833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:39:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:39:36.024441096 +0000 UTC m=+2458.627993704" watchObservedRunningTime="2026-07-15 05:39:36.026647833 +0000 UTC m=+2458.630200444"
Jul 15 05:39:39.212421 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:39.212354    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:39.311175 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:39.311136    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:45.061947 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:45.061909    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:45.062579 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:45.062053    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:45.075127 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:45.075098    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:46.062179 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:46.062146    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:39:46.393025 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:46.392936    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:46.493543 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:46.493508    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:39:47.320859 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:39:47.320822    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:40:09.929182 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:09.929145    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"]
Jul 15 05:40:09.932787 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:09.932761    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:09.935546 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:09.935521    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-de675f\"/\"default-dockercfg-z75cb\""
Jul 15 05:40:09.936439 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:09.936421    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-de675f\"/\"openshift-service-ca.crt\""
Jul 15 05:40:09.936537 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:09.936420    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-de675f\"/\"kube-root-ca.crt\""
Jul 15 05:40:09.936537 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:09.936426    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-de675f\"/\"test-model-de675f-kserve-self-signed-certs\""
Jul 15 05:40:09.941920 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:09.941896    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"]
Jul 15 05:40:09.959571 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:09.959538    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:40:10.088933 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.088894    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-model-cache\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.089130 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.089028    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjnmz\" (UniqueName: \"kubernetes.io/projected/d23eb498-db2f-43be-a743-c5657c89602b-kube-api-access-qjnmz\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.089130 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.089065    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d23eb498-db2f-43be-a743-c5657c89602b-tls-certs\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.089130 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.089095    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-tmp-dir\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.089130 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.089118    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-home\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.089306 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.089151    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-dshm\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.190752 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.190645    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-tmp-dir\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.190752 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.190691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-home\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.190752 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.190709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-dshm\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.190752 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.190743    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-model-cache\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.191094 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.190807    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjnmz\" (UniqueName: \"kubernetes.io/projected/d23eb498-db2f-43be-a743-c5657c89602b-kube-api-access-qjnmz\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.191094 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.190826    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d23eb498-db2f-43be-a743-c5657c89602b-tls-certs\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.191198 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.191154    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-tmp-dir\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.191198 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.191169    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-model-cache\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.191273 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.191225    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-home\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.193073 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.193054    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-dshm\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.193359 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.193341    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d23eb498-db2f-43be-a743-c5657c89602b-tls-certs\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.198306 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.198278    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjnmz\" (UniqueName: \"kubernetes.io/projected/d23eb498-db2f-43be-a743-c5657c89602b-kube-api-access-qjnmz\") pod \"test-model-de675f-kserve-7cd5db4bc5-vdc55\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") " pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.242349 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.242305    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:10.381988 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:10.381961    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"]
Jul 15 05:40:10.384812 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:40:10.384785    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd23eb498_db2f_43be_a743_c5657c89602b.slice/crio-6b1d89588b60b32ceb5204e5d0d4d28ab1722b0b6684a131ed550984b2176891 WatchSource:0}: Error finding container 6b1d89588b60b32ceb5204e5d0d4d28ab1722b0b6684a131ed550984b2176891: Status 404 returned error can't find the container with id 6b1d89588b60b32ceb5204e5d0d4d28ab1722b0b6684a131ed550984b2176891
Jul 15 05:40:11.171581 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:11.171536    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55" event={"ID":"d23eb498-db2f-43be-a743-c5657c89602b","Type":"ContainerStarted","Data":"b4b11aabf767a7ebf2571b243f53a5770dd82325e131a5019189010d141d5cc0"}
Jul 15 05:40:11.171581 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:11.171589    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55" event={"ID":"d23eb498-db2f-43be-a743-c5657c89602b","Type":"ContainerStarted","Data":"6b1d89588b60b32ceb5204e5d0d4d28ab1722b0b6684a131ed550984b2176891"}
Jul 15 05:40:11.189809 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:11.189755    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55" podStartSLOduration=2.189737591 podStartE2EDuration="2.189737591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:40:11.188404428 +0000 UTC m=+2493.791957033" watchObservedRunningTime="2026-07-15 05:40:11.189737591 +0000 UTC m=+2493.793290200"
Jul 15 05:40:20.243097 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:20.243054    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:20.243624 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:20.243114    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:20.256142 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:20.256116    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:21.227787 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:21.227751    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:38.488963 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:38.488853    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:40:39.594810 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:39.594770    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"]
Jul 15 05:40:39.620748 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:39.620716    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:40:40.754398 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:40.754349    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"]
Jul 15 05:40:40.772125 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:40.772089    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:40:41.529547 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:41.529506    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:40:41.631162 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:41.631127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:40:51.332805 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:51.332771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:40:54.615236 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:54.615167    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8" podUID="039e0b63-fbdb-425f-985f-6d6b1f1adc52" containerName="main" containerID="cri-o://6dcf3d74139733c8993d81e26f998e4f8921c5bce5cdcb1a70f1cd3dbc1cdcde" gracePeriod=45
Jul 15 05:40:54.863969 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:54.863940    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:40:54.998795 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:54.998760    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-model-cache\") pod \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") "
Jul 15 05:40:54.998995 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:54.998844    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-tmp-dir\") pod \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") "
Jul 15 05:40:54.998995 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:54.998864    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dpdh\" (UniqueName: \"kubernetes.io/projected/039e0b63-fbdb-425f-985f-6d6b1f1adc52-kube-api-access-8dpdh\") pod \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") "
Jul 15 05:40:54.998995 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:54.998893    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-home\") pod \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") "
Jul 15 05:40:54.998995 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:54.998910    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-dshm\") pod \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") "
Jul 15 05:40:54.998995 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:54.998949    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039e0b63-fbdb-425f-985f-6d6b1f1adc52-tls-certs\") pod \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\" (UID: \"039e0b63-fbdb-425f-985f-6d6b1f1adc52\") "
Jul 15 05:40:54.999250 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:54.999074    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-model-cache" (OuterVolumeSpecName: "model-cache") pod "039e0b63-fbdb-425f-985f-6d6b1f1adc52" (UID: "039e0b63-fbdb-425f-985f-6d6b1f1adc52"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:40:54.999250 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:54.999148    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "039e0b63-fbdb-425f-985f-6d6b1f1adc52" (UID: "039e0b63-fbdb-425f-985f-6d6b1f1adc52"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:40:54.999250 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:54.999214    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-home" (OuterVolumeSpecName: "home") pod "039e0b63-fbdb-425f-985f-6d6b1f1adc52" (UID: "039e0b63-fbdb-425f-985f-6d6b1f1adc52"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:40:54.999348 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:54.999264    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-tmp-dir\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:40:54.999348 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:54.999277    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-model-cache\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:40:55.001257 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.001224    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-dshm" (OuterVolumeSpecName: "dshm") pod "039e0b63-fbdb-425f-985f-6d6b1f1adc52" (UID: "039e0b63-fbdb-425f-985f-6d6b1f1adc52"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:40:55.001414 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.001273    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/039e0b63-fbdb-425f-985f-6d6b1f1adc52-kube-api-access-8dpdh" (OuterVolumeSpecName: "kube-api-access-8dpdh") pod "039e0b63-fbdb-425f-985f-6d6b1f1adc52" (UID: "039e0b63-fbdb-425f-985f-6d6b1f1adc52"). InnerVolumeSpecName "kube-api-access-8dpdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:40:55.001414 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.001271    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/039e0b63-fbdb-425f-985f-6d6b1f1adc52-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "039e0b63-fbdb-425f-985f-6d6b1f1adc52" (UID: "039e0b63-fbdb-425f-985f-6d6b1f1adc52"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:40:55.100753 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.100723    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dpdh\" (UniqueName: \"kubernetes.io/projected/039e0b63-fbdb-425f-985f-6d6b1f1adc52-kube-api-access-8dpdh\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:40:55.100753 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.100749    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-home\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:40:55.100753 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.100758    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039e0b63-fbdb-425f-985f-6d6b1f1adc52-dshm\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:40:55.100971 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.100766    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039e0b63-fbdb-425f-985f-6d6b1f1adc52-tls-certs\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:40:55.369092 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.368998    2562 generic.go:356] "Generic (PLEG): container finished" podID="039e0b63-fbdb-425f-985f-6d6b1f1adc52" containerID="6dcf3d74139733c8993d81e26f998e4f8921c5bce5cdcb1a70f1cd3dbc1cdcde" exitCode=0
Jul 15 05:40:55.369092 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.369037    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"
Jul 15 05:40:55.369092 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.369081    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8" event={"ID":"039e0b63-fbdb-425f-985f-6d6b1f1adc52","Type":"ContainerDied","Data":"6dcf3d74139733c8993d81e26f998e4f8921c5bce5cdcb1a70f1cd3dbc1cdcde"}
Jul 15 05:40:55.369359 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.369123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8" event={"ID":"039e0b63-fbdb-425f-985f-6d6b1f1adc52","Type":"ContainerDied","Data":"703c7c4a768c5eafe57e33010715c0266d7757ff7fc64999c4366083ba3f455e"}
Jul 15 05:40:55.369359 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.369140    2562 scope.go:117] "RemoveContainer" containerID="6dcf3d74139733c8993d81e26f998e4f8921c5bce5cdcb1a70f1cd3dbc1cdcde"
Jul 15 05:40:55.378266 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.378247    2562 scope.go:117] "RemoveContainer" containerID="6dcf3d74139733c8993d81e26f998e4f8921c5bce5cdcb1a70f1cd3dbc1cdcde"
Jul 15 05:40:55.378559 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:40:55.378540    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dcf3d74139733c8993d81e26f998e4f8921c5bce5cdcb1a70f1cd3dbc1cdcde\": container with ID starting with 6dcf3d74139733c8993d81e26f998e4f8921c5bce5cdcb1a70f1cd3dbc1cdcde not found: ID does not exist" containerID="6dcf3d74139733c8993d81e26f998e4f8921c5bce5cdcb1a70f1cd3dbc1cdcde"
Jul 15 05:40:55.378611 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.378569    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dcf3d74139733c8993d81e26f998e4f8921c5bce5cdcb1a70f1cd3dbc1cdcde"} err="failed to get container status \"6dcf3d74139733c8993d81e26f998e4f8921c5bce5cdcb1a70f1cd3dbc1cdcde\": rpc error: code = NotFound desc = could not find container \"6dcf3d74139733c8993d81e26f998e4f8921c5bce5cdcb1a70f1cd3dbc1cdcde\": container with ID starting with 6dcf3d74139733c8993d81e26f998e4f8921c5bce5cdcb1a70f1cd3dbc1cdcde not found: ID does not exist"
Jul 15 05:40:55.389935 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.389907    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"]
Jul 15 05:40:55.392867 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.392845    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-891b7e/test-model-891b7e-kserve-5f565cd585-vwkh8"]
Jul 15 05:40:55.775298 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.775241    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55" podUID="d23eb498-db2f-43be-a743-c5657c89602b" containerName="main" containerID="cri-o://b4b11aabf767a7ebf2571b243f53a5770dd82325e131a5019189010d141d5cc0" gracePeriod=45
Jul 15 05:40:55.844921 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:55.844886    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="039e0b63-fbdb-425f-985f-6d6b1f1adc52" path="/var/lib/kubelet/pods/039e0b63-fbdb-425f-985f-6d6b1f1adc52/volumes"
Jul 15 05:40:56.003197 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.003167    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:40:56.035137 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.035059    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:56.211060 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.211027    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-model-cache\") pod \"d23eb498-db2f-43be-a743-c5657c89602b\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") "
Jul 15 05:40:56.211060 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.211061    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjnmz\" (UniqueName: \"kubernetes.io/projected/d23eb498-db2f-43be-a743-c5657c89602b-kube-api-access-qjnmz\") pod \"d23eb498-db2f-43be-a743-c5657c89602b\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") "
Jul 15 05:40:56.211320 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.211083    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-dshm\") pod \"d23eb498-db2f-43be-a743-c5657c89602b\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") "
Jul 15 05:40:56.211320 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.211118    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-home\") pod \"d23eb498-db2f-43be-a743-c5657c89602b\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") "
Jul 15 05:40:56.211320 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.211159    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d23eb498-db2f-43be-a743-c5657c89602b-tls-certs\") pod \"d23eb498-db2f-43be-a743-c5657c89602b\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") "
Jul 15 05:40:56.211320 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.211173    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-tmp-dir\") pod \"d23eb498-db2f-43be-a743-c5657c89602b\" (UID: \"d23eb498-db2f-43be-a743-c5657c89602b\") "
Jul 15 05:40:56.211560 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.211407    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-model-cache" (OuterVolumeSpecName: "model-cache") pod "d23eb498-db2f-43be-a743-c5657c89602b" (UID: "d23eb498-db2f-43be-a743-c5657c89602b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:40:56.211560 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.211448    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-home" (OuterVolumeSpecName: "home") pod "d23eb498-db2f-43be-a743-c5657c89602b" (UID: "d23eb498-db2f-43be-a743-c5657c89602b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:40:56.211560 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.211525    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d23eb498-db2f-43be-a743-c5657c89602b" (UID: "d23eb498-db2f-43be-a743-c5657c89602b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:40:56.213356 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.213334    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-dshm" (OuterVolumeSpecName: "dshm") pod "d23eb498-db2f-43be-a743-c5657c89602b" (UID: "d23eb498-db2f-43be-a743-c5657c89602b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:40:56.213547 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.213524    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d23eb498-db2f-43be-a743-c5657c89602b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d23eb498-db2f-43be-a743-c5657c89602b" (UID: "d23eb498-db2f-43be-a743-c5657c89602b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:40:56.213547 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.213535    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d23eb498-db2f-43be-a743-c5657c89602b-kube-api-access-qjnmz" (OuterVolumeSpecName: "kube-api-access-qjnmz") pod "d23eb498-db2f-43be-a743-c5657c89602b" (UID: "d23eb498-db2f-43be-a743-c5657c89602b"). InnerVolumeSpecName "kube-api-access-qjnmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:40:56.312468 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.312330    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-model-cache\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:40:56.312468 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.312362    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjnmz\" (UniqueName: \"kubernetes.io/projected/d23eb498-db2f-43be-a743-c5657c89602b-kube-api-access-qjnmz\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:40:56.312468 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.312412    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-dshm\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:40:56.312468 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.312423    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-home\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:40:56.312468 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.312433    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d23eb498-db2f-43be-a743-c5657c89602b-tls-certs\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:40:56.312468 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.312442    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d23eb498-db2f-43be-a743-c5657c89602b-tmp-dir\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:40:56.376150 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.376111    2562 generic.go:356] "Generic (PLEG): container finished" podID="d23eb498-db2f-43be-a743-c5657c89602b" containerID="b4b11aabf767a7ebf2571b243f53a5770dd82325e131a5019189010d141d5cc0" exitCode=0
Jul 15 05:40:56.376150 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.376155    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"
Jul 15 05:40:56.376438 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.376197    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55" event={"ID":"d23eb498-db2f-43be-a743-c5657c89602b","Type":"ContainerDied","Data":"b4b11aabf767a7ebf2571b243f53a5770dd82325e131a5019189010d141d5cc0"}
Jul 15 05:40:56.376438 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.376235    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55" event={"ID":"d23eb498-db2f-43be-a743-c5657c89602b","Type":"ContainerDied","Data":"6b1d89588b60b32ceb5204e5d0d4d28ab1722b0b6684a131ed550984b2176891"}
Jul 15 05:40:56.376438 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.376254    2562 scope.go:117] "RemoveContainer" containerID="b4b11aabf767a7ebf2571b243f53a5770dd82325e131a5019189010d141d5cc0"
Jul 15 05:40:56.385507 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.385490    2562 scope.go:117] "RemoveContainer" containerID="b4b11aabf767a7ebf2571b243f53a5770dd82325e131a5019189010d141d5cc0"
Jul 15 05:40:56.385802 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:40:56.385781    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4b11aabf767a7ebf2571b243f53a5770dd82325e131a5019189010d141d5cc0\": container with ID starting with b4b11aabf767a7ebf2571b243f53a5770dd82325e131a5019189010d141d5cc0 not found: ID does not exist" containerID="b4b11aabf767a7ebf2571b243f53a5770dd82325e131a5019189010d141d5cc0"
Jul 15 05:40:56.385853 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.385810    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4b11aabf767a7ebf2571b243f53a5770dd82325e131a5019189010d141d5cc0"} err="failed to get container status \"b4b11aabf767a7ebf2571b243f53a5770dd82325e131a5019189010d141d5cc0\": rpc error: code = NotFound desc = could not find container \"b4b11aabf767a7ebf2571b243f53a5770dd82325e131a5019189010d141d5cc0\": container with ID starting with b4b11aabf767a7ebf2571b243f53a5770dd82325e131a5019189010d141d5cc0 not found: ID does not exist"
Jul 15 05:40:56.398271 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.398234    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"]
Jul 15 05:40:56.401420 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:56.401394    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-de675f/test-model-de675f-kserve-7cd5db4bc5-vdc55"]
Jul 15 05:40:57.844885 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:40:57.844851    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d23eb498-db2f-43be-a743-c5657c89602b" path="/var/lib/kubelet/pods/d23eb498-db2f-43be-a743-c5657c89602b/volumes"
Jul 15 05:41:01.769176 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:01.769141    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:02.079972 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:02.079885    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:06.702456 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:06.702420    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:06.946199 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:06.946164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:07.541835 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:07.541799    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:07.645618 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:07.645586    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:17.537751 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:17.537677    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:19.940228 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:19.940194    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:22.158566 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:22.158516    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:27.885919 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:27.885874    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:27.986909 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:27.986871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:32.878410 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:32.878380    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:33.101207 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:33.101166    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:33.456351 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:33.456305    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:33.768486 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:33.768361    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:34.066594 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:34.066486    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:36.771654 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:36.771623    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:36.844097 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:36.844064    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:36.903624 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:36.903589    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:37.197235 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:37.197210    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:37.416326 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:37.416294    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:37.516416 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:37.516304    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:39.466730 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:39.466699    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:39.565230 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:39.565197    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:41.403036 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:41.402972    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:41.767480 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:41.767448    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:42.081878 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:42.081781    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:42.227603 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:42.227566    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:44.764890 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:44.764844    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:44.868201 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:44.868158    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:44.963673 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:44.963642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:45.067236 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:45.067149    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:45.599437 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:45.599399    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:45.696716 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:45.696674    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:49.260792 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:49.260757    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:57.409459 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.409414    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj"]
Jul 15 05:41:57.410043 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.409917    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="039e0b63-fbdb-425f-985f-6d6b1f1adc52" containerName="main"
Jul 15 05:41:57.410043 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.409940    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="039e0b63-fbdb-425f-985f-6d6b1f1adc52" containerName="main"
Jul 15 05:41:57.410043 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.409955    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23eb498-db2f-43be-a743-c5657c89602b" containerName="main"
Jul 15 05:41:57.410043 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.409964    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23eb498-db2f-43be-a743-c5657c89602b" containerName="main"
Jul 15 05:41:57.410043 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.410038    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="039e0b63-fbdb-425f-985f-6d6b1f1adc52" containerName="main"
Jul 15 05:41:57.410304 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.410050    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23eb498-db2f-43be-a743-c5657c89602b" containerName="main"
Jul 15 05:41:57.412679 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.412657    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj"
Jul 15 05:41:57.414937 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.414915    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 05:41:57.415074 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.415048    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2zt2\""
Jul 15 05:41:57.419060 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.419037    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj"]
Jul 15 05:41:57.453262 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.453221    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf7892a2-4fab-44f4-a2af-6ab6509ee758-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj\" (UID: \"cf7892a2-4fab-44f4-a2af-6ab6509ee758\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj"
Jul 15 05:41:57.554623 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.554582    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf7892a2-4fab-44f4-a2af-6ab6509ee758-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj\" (UID: \"cf7892a2-4fab-44f4-a2af-6ab6509ee758\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj"
Jul 15 05:41:57.555238 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.555216    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf7892a2-4fab-44f4-a2af-6ab6509ee758-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj\" (UID: \"cf7892a2-4fab-44f4-a2af-6ab6509ee758\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj"
Jul 15 05:41:57.724446 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.724389    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj"
Jul 15 05:41:57.860585 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.860561    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj"]
Jul 15 05:41:57.862661 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:41:57.862633    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7892a2_4fab_44f4_a2af_6ab6509ee758.slice/crio-3219bc09e82b4468d3452c67659f6656c139bce221b2a0346a205a47cbf7f5c8 WatchSource:0}: Error finding container 3219bc09e82b4468d3452c67659f6656c139bce221b2a0346a205a47cbf7f5c8: Status 404 returned error can't find the container with id 3219bc09e82b4468d3452c67659f6656c139bce221b2a0346a205a47cbf7f5c8
Jul 15 05:41:57.864606 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.864589    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 05:41:57.963839 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:57.963805    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:58.064349 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:58.064253    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:41:58.664287 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:41:58.664220    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj" event={"ID":"cf7892a2-4fab-44f4-a2af-6ab6509ee758","Type":"ContainerStarted","Data":"3219bc09e82b4468d3452c67659f6656c139bce221b2a0346a205a47cbf7f5c8"}
Jul 15 05:42:00.675862 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:00.675823    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj" event={"ID":"cf7892a2-4fab-44f4-a2af-6ab6509ee758","Type":"ContainerStarted","Data":"729433ba533ba2c17ede1b55fdeb738301134397e63cc4ac586cdeded41334dc"}
Jul 15 05:42:01.681275 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:01.681234    2562 generic.go:356] "Generic (PLEG): container finished" podID="cf7892a2-4fab-44f4-a2af-6ab6509ee758" containerID="729433ba533ba2c17ede1b55fdeb738301134397e63cc4ac586cdeded41334dc" exitCode=0
Jul 15 05:42:01.681687 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:01.681318    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj" event={"ID":"cf7892a2-4fab-44f4-a2af-6ab6509ee758","Type":"ContainerDied","Data":"729433ba533ba2c17ede1b55fdeb738301134397e63cc4ac586cdeded41334dc"}
Jul 15 05:42:02.818930 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:02.818903    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj"
Jul 15 05:42:02.906219 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:02.906181    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf7892a2-4fab-44f4-a2af-6ab6509ee758-maas-api-service-ca\") pod \"cf7892a2-4fab-44f4-a2af-6ab6509ee758\" (UID: \"cf7892a2-4fab-44f4-a2af-6ab6509ee758\") "
Jul 15 05:42:02.906592 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:02.906565    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf7892a2-4fab-44f4-a2af-6ab6509ee758-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cf7892a2-4fab-44f4-a2af-6ab6509ee758" (UID: "cf7892a2-4fab-44f4-a2af-6ab6509ee758"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:42:03.007166 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:03.007130    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf7892a2-4fab-44f4-a2af-6ab6509ee758-maas-api-service-ca\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:42:03.691102 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:03.691063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj" event={"ID":"cf7892a2-4fab-44f4-a2af-6ab6509ee758","Type":"ContainerDied","Data":"3219bc09e82b4468d3452c67659f6656c139bce221b2a0346a205a47cbf7f5c8"}
Jul 15 05:42:03.691287 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:03.691116    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3219bc09e82b4468d3452c67659f6656c139bce221b2a0346a205a47cbf7f5c8"
Jul 15 05:42:03.691287 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:03.691079    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj"
Jul 15 05:42:06.236159 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:06.236104    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:06.336967 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:06.336931    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:08.160211 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:08.160164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:08.216049 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:08.216017    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:08.296684 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:08.296650    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:12.553316 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:12.553282    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:18.097466 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:18.097410    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:18.195704 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:18.195669    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:23.246490 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.246459    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:23.466108 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.466074    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:23.591024 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.590942    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq"]
Jul 15 05:42:23.591265 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.591253    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf7892a2-4fab-44f4-a2af-6ab6509ee758" containerName="revoke-keys"
Jul 15 05:42:23.591307 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.591266    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf7892a2-4fab-44f4-a2af-6ab6509ee758" containerName="revoke-keys"
Jul 15 05:42:23.591338 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.591333    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf7892a2-4fab-44f4-a2af-6ab6509ee758" containerName="revoke-keys"
Jul 15 05:42:23.594292 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.594273    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq"
Jul 15 05:42:23.596692 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.596665    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2zt2\""
Jul 15 05:42:23.596692 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.596665    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 05:42:23.600833 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.600807    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq"]
Jul 15 05:42:23.700733 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.700692    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13c0d7fd-76c3-42bb-8438-97bddc5ecb81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq\" (UID: \"13c0d7fd-76c3-42bb-8438-97bddc5ecb81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq"
Jul 15 05:42:23.802149 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.802106    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13c0d7fd-76c3-42bb-8438-97bddc5ecb81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq\" (UID: \"13c0d7fd-76c3-42bb-8438-97bddc5ecb81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq"
Jul 15 05:42:23.802802 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.802780    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13c0d7fd-76c3-42bb-8438-97bddc5ecb81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq\" (UID: \"13c0d7fd-76c3-42bb-8438-97bddc5ecb81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq"
Jul 15 05:42:23.906022 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:23.905933    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq"
Jul 15 05:42:24.045492 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:24.045468    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq"]
Jul 15 05:42:24.047883 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:42:24.047855    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13c0d7fd_76c3_42bb_8438_97bddc5ecb81.slice/crio-7306cadff840e77bbc8b367cec03f9c94fe28d3a3cda269ead8cb57133cf0b97 WatchSource:0}: Error finding container 7306cadff840e77bbc8b367cec03f9c94fe28d3a3cda269ead8cb57133cf0b97: Status 404 returned error can't find the container with id 7306cadff840e77bbc8b367cec03f9c94fe28d3a3cda269ead8cb57133cf0b97
Jul 15 05:42:24.063641 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:24.063611    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:24.321940 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:24.321910    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:24.791197 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:24.791158    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq" event={"ID":"13c0d7fd-76c3-42bb-8438-97bddc5ecb81","Type":"ContainerStarted","Data":"e6c8d78e4b14a98ce51932d20f1b63ceaeac9cda2230945dd7598902779fff21"}
Jul 15 05:42:24.791359 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:24.791224    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq" event={"ID":"13c0d7fd-76c3-42bb-8438-97bddc5ecb81","Type":"ContainerStarted","Data":"7306cadff840e77bbc8b367cec03f9c94fe28d3a3cda269ead8cb57133cf0b97"}
Jul 15 05:42:24.805399 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:24.805327    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq" podStartSLOduration=1.805311637 podStartE2EDuration="1.805311637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:42:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:42:24.804593624 +0000 UTC m=+2627.408146234" watchObservedRunningTime="2026-07-15 05:42:24.805311637 +0000 UTC m=+2627.408864245"
Jul 15 05:42:25.796816 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:25.796781    2562 generic.go:356] "Generic (PLEG): container finished" podID="13c0d7fd-76c3-42bb-8438-97bddc5ecb81" containerID="e6c8d78e4b14a98ce51932d20f1b63ceaeac9cda2230945dd7598902779fff21" exitCode=0
Jul 15 05:42:25.797185 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:25.796864    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq" event={"ID":"13c0d7fd-76c3-42bb-8438-97bddc5ecb81","Type":"ContainerDied","Data":"e6c8d78e4b14a98ce51932d20f1b63ceaeac9cda2230945dd7598902779fff21"}
Jul 15 05:42:26.940779 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:26.940750    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq"
Jul 15 05:42:27.030946 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:27.030905    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13c0d7fd-76c3-42bb-8438-97bddc5ecb81-maas-api-service-ca\") pod \"13c0d7fd-76c3-42bb-8438-97bddc5ecb81\" (UID: \"13c0d7fd-76c3-42bb-8438-97bddc5ecb81\") "
Jul 15 05:42:27.031271 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:27.031247    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13c0d7fd-76c3-42bb-8438-97bddc5ecb81-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "13c0d7fd-76c3-42bb-8438-97bddc5ecb81" (UID: "13c0d7fd-76c3-42bb-8438-97bddc5ecb81"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:42:27.131794 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:27.131701    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13c0d7fd-76c3-42bb-8438-97bddc5ecb81-maas-api-service-ca\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:42:27.810021 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:27.809989    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq" event={"ID":"13c0d7fd-76c3-42bb-8438-97bddc5ecb81","Type":"ContainerDied","Data":"7306cadff840e77bbc8b367cec03f9c94fe28d3a3cda269ead8cb57133cf0b97"}
Jul 15 05:42:27.810217 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:27.810028    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq"
Jul 15 05:42:27.810217 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:27.810039    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7306cadff840e77bbc8b367cec03f9c94fe28d3a3cda269ead8cb57133cf0b97"
Jul 15 05:42:34.014038 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:34.014006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:37.960633 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:37.960595    2562 scope.go:117] "RemoveContainer" containerID="5e0d8d2a8a5c4f741c7a59598922ce1ecbb5bbebb54c20db77c6695765475676"
Jul 15 05:42:38.672418 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:38.672357    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:44.301130 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:44.301094    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:44.402117 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:44.402086    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:49.374778 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:49.374746    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:49.632669 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:49.632547    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:49.966102 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:49.966065    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:51.235794 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:51.235763    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:51.333734 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:51.333694    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:53.178732 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:53.178699    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:53.274143 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:53.274102    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:53.459111 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:53.459081    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:53.809910 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:53.809822    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:54.029228 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:54.029182    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:54.134052 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:54.133956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:57.691733 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:57.691699    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:58.012464 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:58.012430    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:58.323102 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:58.323010    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:42:58.480703 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:42:58.480666    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:01.285539 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:01.285506    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:01.388575 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:01.388539    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:01.480696 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:01.480660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:01.673382 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:01.673290    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:01.937089 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:01.937011    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:02.038579 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:02.038549    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:05.739964 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:05.739931    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:14.816111 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:14.816041    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:14.916017 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:14.915980    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:18.816511 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.816476    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"]
Jul 15 05:43:18.816980 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.816795    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13c0d7fd-76c3-42bb-8438-97bddc5ecb81" containerName="revoke-keys"
Jul 15 05:43:18.816980 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.816805    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c0d7fd-76c3-42bb-8438-97bddc5ecb81" containerName="revoke-keys"
Jul 15 05:43:18.816980 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.816864    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="13c0d7fd-76c3-42bb-8438-97bddc5ecb81" containerName="revoke-keys"
Jul 15 05:43:18.819654 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.819636    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:18.822351 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.822327    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d43b4e\"/\"kube-root-ca.crt\""
Jul 15 05:43:18.822490 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.822414    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d43b4e\"/\"default-dockercfg-h2rd9\""
Jul 15 05:43:18.823520 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.823498    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d43b4e\"/\"auth-test-model-d43b4e-kserve-self-signed-certs\""
Jul 15 05:43:18.823623 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.823526    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d43b4e\"/\"openshift-service-ca.crt\""
Jul 15 05:43:18.828574 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.828554    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"]
Jul 15 05:43:18.840779 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.840748    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:18.999270 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.999231    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpn7q\" (UniqueName: \"kubernetes.io/projected/3eab9065-be03-4432-af87-5e49d5b0dd66-kube-api-access-tpn7q\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:18.999270 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.999273    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-tmp-dir\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:18.999528 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.999292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3eab9065-be03-4432-af87-5e49d5b0dd66-tls-certs\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:18.999528 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.999310    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-model-cache\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:18.999528 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.999431    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-home\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:18.999528 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:18.999466    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-dshm\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.099959 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.099853    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpn7q\" (UniqueName: \"kubernetes.io/projected/3eab9065-be03-4432-af87-5e49d5b0dd66-kube-api-access-tpn7q\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.099959 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.099906    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-tmp-dir\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.100197 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.100020    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3eab9065-be03-4432-af87-5e49d5b0dd66-tls-certs\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.100197 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.100064    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-model-cache\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.100197 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.100151    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-home\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.100197 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.100181    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-dshm\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.100524 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.100308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-tmp-dir\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.100524 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.100461    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-model-cache\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.100524 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.100499    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-home\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.102337 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.102317    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-dshm\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.102649 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.102632    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3eab9065-be03-4432-af87-5e49d5b0dd66-tls-certs\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.107423 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.107406    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpn7q\" (UniqueName: \"kubernetes.io/projected/3eab9065-be03-4432-af87-5e49d5b0dd66-kube-api-access-tpn7q\") pod \"auth-test-model-d43b4e-kserve-784d77ddc5-42w96\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") " pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.130487 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.130450    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:19.270708 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:19.270677    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"]
Jul 15 05:43:19.273538 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:43:19.273505    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3eab9065_be03_4432_af87_5e49d5b0dd66.slice/crio-acd491c3d1e8a9302dfef3f6ff279a0440e32c43b75b07696e52e7b18c4678cf WatchSource:0}: Error finding container acd491c3d1e8a9302dfef3f6ff279a0440e32c43b75b07696e52e7b18c4678cf: Status 404 returned error can't find the container with id acd491c3d1e8a9302dfef3f6ff279a0440e32c43b75b07696e52e7b18c4678cf
Jul 15 05:43:20.042416 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:20.042364    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96" event={"ID":"3eab9065-be03-4432-af87-5e49d5b0dd66","Type":"ContainerStarted","Data":"72a56bc6b2a2814200639dad7b1804e7b754ac282f1d6c52a163cbf69a41d11b"}
Jul 15 05:43:20.042416 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:20.042422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96" event={"ID":"3eab9065-be03-4432-af87-5e49d5b0dd66","Type":"ContainerStarted","Data":"acd491c3d1e8a9302dfef3f6ff279a0440e32c43b75b07696e52e7b18c4678cf"}
Jul 15 05:43:20.057932 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:20.057876    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96" podStartSLOduration=2.057858771 podStartE2EDuration="2.057858771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:43:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:43:20.057292908 +0000 UTC m=+2682.660845517" watchObservedRunningTime="2026-07-15 05:43:20.057858771 +0000 UTC m=+2682.661411405"
Jul 15 05:43:22.666465 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:22.666432    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:23.756113 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:23.756044    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:27.941412 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:27.941364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:28.039777 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:28.039740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:29.130842 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:29.130802    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:29.131232 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:29.130857    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:29.143159 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:29.143134    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:30.097254 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:30.097179    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:43:38.338980 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:38.338867    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:43:38.344978 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:38.344956    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:43:42.689155 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.689116    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"]
Jul 15 05:43:42.692549 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.692518    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.696360 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.696311    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1edf20\"/\"openshift-service-ca.crt\""
Jul 15 05:43:42.696360 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.696311    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1edf20\"/\"default-dockercfg-mp96r\""
Jul 15 05:43:42.696360 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.696357    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1edf20\"/\"kube-root-ca.crt\""
Jul 15 05:43:42.696665 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.696393    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1edf20\"/\"auth-test-model-1edf20-kserve-self-signed-certs\""
Jul 15 05:43:42.705041 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.704999    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"]
Jul 15 05:43:42.738055 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.738018    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:43:42.808111 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.808070    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadb6908-f0ac-46b7-9c04-3119c389320e-tls-certs\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.808111 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.808113    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-home\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.808336 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.808141    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-tmp-dir\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.808336 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.808196    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-model-cache\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.808336 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.808284    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w999\" (UniqueName: \"kubernetes.io/projected/fadb6908-f0ac-46b7-9c04-3119c389320e-kube-api-access-4w999\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.808336 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.808307    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-dshm\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.909266 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.909214    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-tmp-dir\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.909266 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.909256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-model-cache\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.909535 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.909322    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w999\" (UniqueName: \"kubernetes.io/projected/fadb6908-f0ac-46b7-9c04-3119c389320e-kube-api-access-4w999\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.909535 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.909343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-dshm\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.909535 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.909432    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadb6908-f0ac-46b7-9c04-3119c389320e-tls-certs\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.909535 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.909462    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-home\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.909725 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.909692    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-tmp-dir\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.909781 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.909749    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-model-cache\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.909781 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.909770    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-home\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.911718 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.911695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-dshm\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.912106 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.912084    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadb6908-f0ac-46b7-9c04-3119c389320e-tls-certs\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:42.917152 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:42.917129    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w999\" (UniqueName: \"kubernetes.io/projected/fadb6908-f0ac-46b7-9c04-3119c389320e-kube-api-access-4w999\") pod \"auth-test-model-1edf20-kserve-74b9bb9595-h7xvw\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") " pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:43.003228 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:43.003186    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:43.137633 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:43.137603    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"]
Jul 15 05:43:43.139269 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:43:43.139241    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfadb6908_f0ac_46b7_9c04_3119c389320e.slice/crio-dc95f47d0a8e327e6a753b3151029e0485205865bfc8948c0d08a7d206d6d4bf WatchSource:0}: Error finding container dc95f47d0a8e327e6a753b3151029e0485205865bfc8948c0d08a7d206d6d4bf: Status 404 returned error can't find the container with id dc95f47d0a8e327e6a753b3151029e0485205865bfc8948c0d08a7d206d6d4bf
Jul 15 05:43:43.148062 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:43.148037    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw" event={"ID":"fadb6908-f0ac-46b7-9c04-3119c389320e","Type":"ContainerStarted","Data":"dc95f47d0a8e327e6a753b3151029e0485205865bfc8948c0d08a7d206d6d4bf"}
Jul 15 05:43:44.154961 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:44.154919    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw" event={"ID":"fadb6908-f0ac-46b7-9c04-3119c389320e","Type":"ContainerStarted","Data":"3a2617944b0ed466a414608619381eb8f87f03cd22e0f20bfd2becadd0cad3c8"}
Jul 15 05:43:44.171351 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:44.171300    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw" podStartSLOduration=2.171284596 podStartE2EDuration="2.171284596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:43:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:43:44.170054059 +0000 UTC m=+2706.773606667" watchObservedRunningTime="2026-07-15 05:43:44.171284596 +0000 UTC m=+2706.774837222"
Jul 15 05:43:53.003830 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:53.003794    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:53.004245 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:53.003847    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:53.016364 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:53.016337    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:43:53.210496 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:43:53.210465    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:45:00.132642 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.132543    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1edf20-29734905-6gx2n"]
Jul 15 05:45:00.135862 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.135839    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734905-kvlbj"]
Jul 15 05:45:00.136015 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.135966    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1edf20-29734905-6gx2n"
Jul 15 05:45:00.138340 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.138320    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2zt2\""
Jul 15 05:45:00.138943 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.138929    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d43b4e-29734905-srb7b"]
Jul 15 05:45:00.139050 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.139028    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734905-kvlbj"
Jul 15 05:45:00.142061 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.142036    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d43b4e-29734905-srb7b"
Jul 15 05:45:00.145541 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.145485    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d43b4e-29734905-srb7b"]
Jul 15 05:45:00.147663 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.147637    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734905-kvlbj"]
Jul 15 05:45:00.147972 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.147952    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1edf20-29734905-6gx2n"
Jul 15 05:45:00.149685 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.149666    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1edf20-29734905-6gx2n"]
Jul 15 05:45:00.157009 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.156989    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734905-kvlbj"
Jul 15 05:45:00.184287 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.184250    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d43b4e-29734905-srb7b"
Jul 15 05:45:00.309031 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.308982    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1edf20-29734905-6gx2n"]
Jul 15 05:45:00.313173 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:45:00.313142    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc85822b6_3f60_4f5e_8dc6_54706e8db05d.slice/crio-255900db49b18d874680a6ad48ec6b83b1d0b081a585f8e507c09bd23186dfab WatchSource:0}: Error finding container 255900db49b18d874680a6ad48ec6b83b1d0b081a585f8e507c09bd23186dfab: Status 404 returned error can't find the container with id 255900db49b18d874680a6ad48ec6b83b1d0b081a585f8e507c09bd23186dfab
Jul 15 05:45:00.345918 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.345890    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734905-kvlbj"]
Jul 15 05:45:00.354766 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:45:00.354739    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f7a50bf_e87d_4261_9e08_739cd95560ef.slice/crio-3470882f528ce2a66f9786e5e83275a153d96062b109657bd129cf9102933c60 WatchSource:0}: Error finding container 3470882f528ce2a66f9786e5e83275a153d96062b109657bd129cf9102933c60: Status 404 returned error can't find the container with id 3470882f528ce2a66f9786e5e83275a153d96062b109657bd129cf9102933c60
Jul 15 05:45:00.366578 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.366554    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d43b4e-29734905-srb7b"]
Jul 15 05:45:00.368836 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:45:00.368807    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode638abc7_e796_4c3e_978b_a783008ee080.slice/crio-bbcfc61012b6dede34313e627af7e8758e3d8f03663f76a337d3b39cbfd0042c WatchSource:0}: Error finding container bbcfc61012b6dede34313e627af7e8758e3d8f03663f76a337d3b39cbfd0042c: Status 404 returned error can't find the container with id bbcfc61012b6dede34313e627af7e8758e3d8f03663f76a337d3b39cbfd0042c
Jul 15 05:45:00.496897 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.496637    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734905-kvlbj" event={"ID":"1f7a50bf-e87d-4261-9e08-739cd95560ef","Type":"ContainerStarted","Data":"3470882f528ce2a66f9786e5e83275a153d96062b109657bd129cf9102933c60"}
Jul 15 05:45:00.498208 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.498183    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1edf20-29734905-6gx2n" event={"ID":"c85822b6-3f60-4f5e-8dc6-54706e8db05d","Type":"ContainerStarted","Data":"255900db49b18d874680a6ad48ec6b83b1d0b081a585f8e507c09bd23186dfab"}
Jul 15 05:45:00.499776 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:00.499745    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d43b4e-29734905-srb7b" event={"ID":"e638abc7-e796-4c3e-978b-a783008ee080","Type":"ContainerStarted","Data":"bbcfc61012b6dede34313e627af7e8758e3d8f03663f76a337d3b39cbfd0042c"}
Jul 15 05:45:01.505422 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:01.505355    2562 generic.go:356] "Generic (PLEG): container finished" podID="e638abc7-e796-4c3e-978b-a783008ee080" containerID="ca0dae6ee751e494361bb57ad7b9df340e5cd7f69070d5deca6cd7b69d01afca" exitCode=0
Jul 15 05:45:01.505834 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:01.505441    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d43b4e-29734905-srb7b" event={"ID":"e638abc7-e796-4c3e-978b-a783008ee080","Type":"ContainerDied","Data":"ca0dae6ee751e494361bb57ad7b9df340e5cd7f69070d5deca6cd7b69d01afca"}
Jul 15 05:45:01.507037 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:01.507015    2562 generic.go:356] "Generic (PLEG): container finished" podID="1f7a50bf-e87d-4261-9e08-739cd95560ef" containerID="5ab384da8bc494deab2197ddf2198defd660ff95af711ab0749b8237458db7fb" exitCode=0
Jul 15 05:45:01.507146 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:01.507082    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734905-kvlbj" event={"ID":"1f7a50bf-e87d-4261-9e08-739cd95560ef","Type":"ContainerDied","Data":"5ab384da8bc494deab2197ddf2198defd660ff95af711ab0749b8237458db7fb"}
Jul 15 05:45:01.508559 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:01.508537    2562 generic.go:356] "Generic (PLEG): container finished" podID="c85822b6-3f60-4f5e-8dc6-54706e8db05d" containerID="f0928e98df42accced2c7bceda450d0b4684189b3cd0da63de452a1338104f09" exitCode=0
Jul 15 05:45:01.508677 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:01.508623    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1edf20-29734905-6gx2n" event={"ID":"c85822b6-3f60-4f5e-8dc6-54706e8db05d","Type":"ContainerDied","Data":"f0928e98df42accced2c7bceda450d0b4684189b3cd0da63de452a1338104f09"}
Jul 15 05:45:02.694793 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:02.694766    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1edf20-29734905-6gx2n"
Jul 15 05:45:02.697635 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:02.697612    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734905-kvlbj"
Jul 15 05:45:02.700804 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:02.700785    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d43b4e-29734905-srb7b"
Jul 15 05:45:03.519847 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:03.519808    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734905-kvlbj"
Jul 15 05:45:03.520058 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:03.519808    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734905-kvlbj" event={"ID":"1f7a50bf-e87d-4261-9e08-739cd95560ef","Type":"ContainerDied","Data":"3470882f528ce2a66f9786e5e83275a153d96062b109657bd129cf9102933c60"}
Jul 15 05:45:03.520058 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:03.519949    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3470882f528ce2a66f9786e5e83275a153d96062b109657bd129cf9102933c60"
Jul 15 05:45:03.521112 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:03.521091    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1edf20-29734905-6gx2n"
Jul 15 05:45:03.521228 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:03.521131    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1edf20-29734905-6gx2n" event={"ID":"c85822b6-3f60-4f5e-8dc6-54706e8db05d","Type":"ContainerDied","Data":"255900db49b18d874680a6ad48ec6b83b1d0b081a585f8e507c09bd23186dfab"}
Jul 15 05:45:03.521228 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:03.521177    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="255900db49b18d874680a6ad48ec6b83b1d0b081a585f8e507c09bd23186dfab"
Jul 15 05:45:03.522347 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:03.522326    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d43b4e-29734905-srb7b" event={"ID":"e638abc7-e796-4c3e-978b-a783008ee080","Type":"ContainerDied","Data":"bbcfc61012b6dede34313e627af7e8758e3d8f03663f76a337d3b39cbfd0042c"}
Jul 15 05:45:03.522347 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:03.522342    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d43b4e-29734905-srb7b"
Jul 15 05:45:03.522495 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:45:03.522391    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbcfc61012b6dede34313e627af7e8758e3d8f03663f76a337d3b39cbfd0042c"
Jul 15 05:46:00.950721 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:00.950686    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:46:13.778784 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:13.778748    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"]
Jul 15 05:46:13.798578 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:13.798543    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:46:14.711582 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:14.711540    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"]
Jul 15 05:46:14.720169 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:14.720133    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:46:18.750080 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:18.750046    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:46:21.171200 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:21.171164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:46:28.799686 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:28.799621    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96" podUID="3eab9065-be03-4432-af87-5e49d5b0dd66" containerName="main" containerID="cri-o://72a56bc6b2a2814200639dad7b1804e7b754ac282f1d6c52a163cbf69a41d11b" gracePeriod=45
Jul 15 05:46:28.916751 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:28.916726    2562 generic.go:356] "Generic (PLEG): container finished" podID="3eab9065-be03-4432-af87-5e49d5b0dd66" containerID="72a56bc6b2a2814200639dad7b1804e7b754ac282f1d6c52a163cbf69a41d11b" exitCode=0
Jul 15 05:46:28.916822 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:28.916804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96" event={"ID":"3eab9065-be03-4432-af87-5e49d5b0dd66","Type":"ContainerDied","Data":"72a56bc6b2a2814200639dad7b1804e7b754ac282f1d6c52a163cbf69a41d11b"}
Jul 15 05:46:29.054989 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.054916    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:46:29.154498 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.154466    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-home\") pod \"3eab9065-be03-4432-af87-5e49d5b0dd66\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") "
Jul 15 05:46:29.154676 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.154537    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-tmp-dir\") pod \"3eab9065-be03-4432-af87-5e49d5b0dd66\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") "
Jul 15 05:46:29.154676 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.154571    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-model-cache\") pod \"3eab9065-be03-4432-af87-5e49d5b0dd66\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") "
Jul 15 05:46:29.154676 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.154605    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpn7q\" (UniqueName: \"kubernetes.io/projected/3eab9065-be03-4432-af87-5e49d5b0dd66-kube-api-access-tpn7q\") pod \"3eab9065-be03-4432-af87-5e49d5b0dd66\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") "
Jul 15 05:46:29.154676 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.154624    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3eab9065-be03-4432-af87-5e49d5b0dd66-tls-certs\") pod \"3eab9065-be03-4432-af87-5e49d5b0dd66\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") "
Jul 15 05:46:29.154676 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.154660    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-dshm\") pod \"3eab9065-be03-4432-af87-5e49d5b0dd66\" (UID: \"3eab9065-be03-4432-af87-5e49d5b0dd66\") "
Jul 15 05:46:29.154942 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.154808    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-home" (OuterVolumeSpecName: "home") pod "3eab9065-be03-4432-af87-5e49d5b0dd66" (UID: "3eab9065-be03-4432-af87-5e49d5b0dd66"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:46:29.154942 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.154864    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-model-cache" (OuterVolumeSpecName: "model-cache") pod "3eab9065-be03-4432-af87-5e49d5b0dd66" (UID: "3eab9065-be03-4432-af87-5e49d5b0dd66"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:46:29.154942 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.154890    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3eab9065-be03-4432-af87-5e49d5b0dd66" (UID: "3eab9065-be03-4432-af87-5e49d5b0dd66"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:46:29.155092 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.154998    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-home\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:46:29.155092 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.155009    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-tmp-dir\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:46:29.155092 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.155019    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-model-cache\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:46:29.156903 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.156873    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eab9065-be03-4432-af87-5e49d5b0dd66-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3eab9065-be03-4432-af87-5e49d5b0dd66" (UID: "3eab9065-be03-4432-af87-5e49d5b0dd66"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:46:29.156903 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.156866    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-dshm" (OuterVolumeSpecName: "dshm") pod "3eab9065-be03-4432-af87-5e49d5b0dd66" (UID: "3eab9065-be03-4432-af87-5e49d5b0dd66"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:46:29.157072 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.156898    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eab9065-be03-4432-af87-5e49d5b0dd66-kube-api-access-tpn7q" (OuterVolumeSpecName: "kube-api-access-tpn7q") pod "3eab9065-be03-4432-af87-5e49d5b0dd66" (UID: "3eab9065-be03-4432-af87-5e49d5b0dd66"). InnerVolumeSpecName "kube-api-access-tpn7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:46:29.255988 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.255942    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpn7q\" (UniqueName: \"kubernetes.io/projected/3eab9065-be03-4432-af87-5e49d5b0dd66-kube-api-access-tpn7q\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:46:29.255988 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.255976    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3eab9065-be03-4432-af87-5e49d5b0dd66-tls-certs\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:46:29.255988 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.255985    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3eab9065-be03-4432-af87-5e49d5b0dd66-dshm\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:46:29.732429 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.732341    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw" podUID="fadb6908-f0ac-46b7-9c04-3119c389320e" containerName="main" containerID="cri-o://3a2617944b0ed466a414608619381eb8f87f03cd22e0f20bfd2becadd0cad3c8" gracePeriod=45
Jul 15 05:46:29.924910 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.924873    2562 generic.go:356] "Generic (PLEG): container finished" podID="fadb6908-f0ac-46b7-9c04-3119c389320e" containerID="3a2617944b0ed466a414608619381eb8f87f03cd22e0f20bfd2becadd0cad3c8" exitCode=0
Jul 15 05:46:29.925325 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.924947    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw" event={"ID":"fadb6908-f0ac-46b7-9c04-3119c389320e","Type":"ContainerDied","Data":"3a2617944b0ed466a414608619381eb8f87f03cd22e0f20bfd2becadd0cad3c8"}
Jul 15 05:46:29.926457 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.926427    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"
Jul 15 05:46:29.926573 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.926461    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96" event={"ID":"3eab9065-be03-4432-af87-5e49d5b0dd66","Type":"ContainerDied","Data":"acd491c3d1e8a9302dfef3f6ff279a0440e32c43b75b07696e52e7b18c4678cf"}
Jul 15 05:46:29.926573 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.926512    2562 scope.go:117] "RemoveContainer" containerID="72a56bc6b2a2814200639dad7b1804e7b754ac282f1d6c52a163cbf69a41d11b"
Jul 15 05:46:29.942314 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.942282    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"]
Jul 15 05:46:29.948020 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.947992    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d43b4e/auth-test-model-d43b4e-kserve-784d77ddc5-42w96"]
Jul 15 05:46:29.983690 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:29.983614    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:46:30.063276 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.063241    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-dshm\") pod \"fadb6908-f0ac-46b7-9c04-3119c389320e\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") "
Jul 15 05:46:30.063494 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.063286    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-home\") pod \"fadb6908-f0ac-46b7-9c04-3119c389320e\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") "
Jul 15 05:46:30.063494 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.063311    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-model-cache\") pod \"fadb6908-f0ac-46b7-9c04-3119c389320e\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") "
Jul 15 05:46:30.063494 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.063336    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadb6908-f0ac-46b7-9c04-3119c389320e-tls-certs\") pod \"fadb6908-f0ac-46b7-9c04-3119c389320e\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") "
Jul 15 05:46:30.063494 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.063362    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4w999\" (UniqueName: \"kubernetes.io/projected/fadb6908-f0ac-46b7-9c04-3119c389320e-kube-api-access-4w999\") pod \"fadb6908-f0ac-46b7-9c04-3119c389320e\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") "
Jul 15 05:46:30.063494 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.063474    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-tmp-dir\") pod \"fadb6908-f0ac-46b7-9c04-3119c389320e\" (UID: \"fadb6908-f0ac-46b7-9c04-3119c389320e\") "
Jul 15 05:46:30.063823 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.063650    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-home" (OuterVolumeSpecName: "home") pod "fadb6908-f0ac-46b7-9c04-3119c389320e" (UID: "fadb6908-f0ac-46b7-9c04-3119c389320e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:46:30.063823 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.063662    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-model-cache" (OuterVolumeSpecName: "model-cache") pod "fadb6908-f0ac-46b7-9c04-3119c389320e" (UID: "fadb6908-f0ac-46b7-9c04-3119c389320e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:46:30.063921 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.063855    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-home\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:46:30.063921 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.063875    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-model-cache\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:46:30.063921 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.063875    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fadb6908-f0ac-46b7-9c04-3119c389320e" (UID: "fadb6908-f0ac-46b7-9c04-3119c389320e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:46:30.065684 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.065650    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fadb6908-f0ac-46b7-9c04-3119c389320e-kube-api-access-4w999" (OuterVolumeSpecName: "kube-api-access-4w999") pod "fadb6908-f0ac-46b7-9c04-3119c389320e" (UID: "fadb6908-f0ac-46b7-9c04-3119c389320e"). InnerVolumeSpecName "kube-api-access-4w999". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:46:30.065928 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.065906    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-dshm" (OuterVolumeSpecName: "dshm") pod "fadb6908-f0ac-46b7-9c04-3119c389320e" (UID: "fadb6908-f0ac-46b7-9c04-3119c389320e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:46:30.065989 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.065921    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fadb6908-f0ac-46b7-9c04-3119c389320e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fadb6908-f0ac-46b7-9c04-3119c389320e" (UID: "fadb6908-f0ac-46b7-9c04-3119c389320e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:46:30.164628 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.164586    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-tmp-dir\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:46:30.164628 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.164619    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadb6908-f0ac-46b7-9c04-3119c389320e-dshm\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:46:30.164628 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.164627    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadb6908-f0ac-46b7-9c04-3119c389320e-tls-certs\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:46:30.164628 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.164638    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4w999\" (UniqueName: \"kubernetes.io/projected/fadb6908-f0ac-46b7-9c04-3119c389320e-kube-api-access-4w999\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:46:30.932762 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.932723    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"
Jul 15 05:46:30.933226 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.932722    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw" event={"ID":"fadb6908-f0ac-46b7-9c04-3119c389320e","Type":"ContainerDied","Data":"dc95f47d0a8e327e6a753b3151029e0485205865bfc8948c0d08a7d206d6d4bf"}
Jul 15 05:46:30.933226 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.932862    2562 scope.go:117] "RemoveContainer" containerID="3a2617944b0ed466a414608619381eb8f87f03cd22e0f20bfd2becadd0cad3c8"
Jul 15 05:46:30.953709 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.953676    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"]
Jul 15 05:46:30.957348 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:30.957318    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/auth-test-model-1edf20-kserve-74b9bb9595-h7xvw"]
Jul 15 05:46:31.845624 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:31.845592    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eab9065-be03-4432-af87-5e49d5b0dd66" path="/var/lib/kubelet/pods/3eab9065-be03-4432-af87-5e49d5b0dd66/volumes"
Jul 15 05:46:31.846006 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:31.845993    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fadb6908-f0ac-46b7-9c04-3119c389320e" path="/var/lib/kubelet/pods/fadb6908-f0ac-46b7-9c04-3119c389320e/volumes"
Jul 15 05:46:40.249870 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:40.249823    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:46:40.350979 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:40.350925    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:46:51.336295 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336250    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"]
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336609    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e638abc7-e796-4c3e-978b-a783008ee080" containerName="cleanup"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336624    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e638abc7-e796-4c3e-978b-a783008ee080" containerName="cleanup"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336636    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fadb6908-f0ac-46b7-9c04-3119c389320e" containerName="main"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336642    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fadb6908-f0ac-46b7-9c04-3119c389320e" containerName="main"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336651    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c85822b6-3f60-4f5e-8dc6-54706e8db05d" containerName="cleanup"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336657    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c85822b6-3f60-4f5e-8dc6-54706e8db05d" containerName="cleanup"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336666    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f7a50bf-e87d-4261-9e08-739cd95560ef" containerName="cleanup"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336670    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f7a50bf-e87d-4261-9e08-739cd95560ef" containerName="cleanup"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336678    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eab9065-be03-4432-af87-5e49d5b0dd66" containerName="main"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336686    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eab9065-be03-4432-af87-5e49d5b0dd66" containerName="main"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336746    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fadb6908-f0ac-46b7-9c04-3119c389320e" containerName="main"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336754    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e638abc7-e796-4c3e-978b-a783008ee080" containerName="cleanup"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336763    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c85822b6-3f60-4f5e-8dc6-54706e8db05d" containerName="cleanup"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336773    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eab9065-be03-4432-af87-5e49d5b0dd66" containerName="main"
Jul 15 05:46:51.336817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.336779    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f7a50bf-e87d-4261-9e08-739cd95560ef" containerName="cleanup"
Jul 15 05:46:51.339695 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.339673    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.343177 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.343151    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1edf20\"/\"default-dockercfg-mp96r\""
Jul 15 05:46:51.343305 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.343179    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1edf20\"/\"openshift-service-ca.crt\""
Jul 15 05:46:51.343305 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.343159    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1edf20\"/\"kube-root-ca.crt\""
Jul 15 05:46:51.343305 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.343224    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1edf20\"/\"sub-test-model-1edf20-kserve-self-signed-certs\""
Jul 15 05:46:51.349242 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.349222    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"]
Jul 15 05:46:51.353602 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.353575    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-model-cache\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.353705 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.353616    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f33bb527-27e4-4724-85cc-3a4d0447eaed-tls-certs\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.353705 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.353634    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-home\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.353705 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.353682    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-tmp-dir\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.354349 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.354315    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6dgj\" (UniqueName: \"kubernetes.io/projected/f33bb527-27e4-4724-85cc-3a4d0447eaed-kube-api-access-q6dgj\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.354600 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.354579    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-dshm\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.395478 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.395442    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:46:51.455738 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.455698    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-model-cache\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.455899 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.455746    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f33bb527-27e4-4724-85cc-3a4d0447eaed-tls-certs\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.455899 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.455771    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-home\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.455899 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.455796    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-tmp-dir\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.455899 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.455832    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6dgj\" (UniqueName: \"kubernetes.io/projected/f33bb527-27e4-4724-85cc-3a4d0447eaed-kube-api-access-q6dgj\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.455899 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.455892    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-dshm\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.456205 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.456179    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-home\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.456273 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.456180    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-model-cache\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.456273 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.456239    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-tmp-dir\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.458184 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.458162    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-dshm\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.458496 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.458473    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f33bb527-27e4-4724-85cc-3a4d0447eaed-tls-certs\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.463843 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.463820    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6dgj\" (UniqueName: \"kubernetes.io/projected/f33bb527-27e4-4724-85cc-3a4d0447eaed-kube-api-access-q6dgj\") pod \"sub-test-model-1edf20-kserve-764f45fc55-sqqs8\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") " pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.650194 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.650100    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:46:51.786694 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:51.786667    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"]
Jul 15 05:46:51.789420 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:46:51.789391    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf33bb527_27e4_4724_85cc_3a4d0447eaed.slice/crio-7154ccea8bc7a74e292a4e348eebaddceb556d1591bee46d21fa80f0e2abb3bc WatchSource:0}: Error finding container 7154ccea8bc7a74e292a4e348eebaddceb556d1591bee46d21fa80f0e2abb3bc: Status 404 returned error can't find the container with id 7154ccea8bc7a74e292a4e348eebaddceb556d1591bee46d21fa80f0e2abb3bc
Jul 15 05:46:52.034315 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:52.034279    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8" event={"ID":"f33bb527-27e4-4724-85cc-3a4d0447eaed","Type":"ContainerStarted","Data":"b37837593adf4994a65e86a045af39ce151796a20d8fab2ab37d3eb38f551b44"}
Jul 15 05:46:52.034533 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:52.034328    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8" event={"ID":"f33bb527-27e4-4724-85cc-3a4d0447eaed","Type":"ContainerStarted","Data":"7154ccea8bc7a74e292a4e348eebaddceb556d1591bee46d21fa80f0e2abb3bc"}
Jul 15 05:46:52.050305 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:46:52.050249    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8" podStartSLOduration=1.050232193 podStartE2EDuration="1.050232193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:46:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:46:52.048784454 +0000 UTC m=+2894.652337098" watchObservedRunningTime="2026-07-15 05:46:52.050232193 +0000 UTC m=+2894.653784801"
Jul 15 05:47:01.650821 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:01.650777    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:47:01.651267 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:01.650877    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:47:01.663406 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:01.663351    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:47:02.092104 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:02.092075    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:47:39.858272 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:39.858239    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:47:40.934323 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:40.934287    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"]
Jul 15 05:47:40.953591 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:40.953559    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:47:55.954782 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:55.954743    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8" podUID="f33bb527-27e4-4724-85cc-3a4d0447eaed" containerName="main" containerID="cri-o://b37837593adf4994a65e86a045af39ce151796a20d8fab2ab37d3eb38f551b44" gracePeriod=45
Jul 15 05:47:56.204466 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.204438    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:47:56.242284 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.242180    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-home\") pod \"f33bb527-27e4-4724-85cc-3a4d0447eaed\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") "
Jul 15 05:47:56.242284 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.242255    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-model-cache\") pod \"f33bb527-27e4-4724-85cc-3a4d0447eaed\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") "
Jul 15 05:47:56.242557 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.242294    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-tmp-dir\") pod \"f33bb527-27e4-4724-85cc-3a4d0447eaed\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") "
Jul 15 05:47:56.242557 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.242317    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6dgj\" (UniqueName: \"kubernetes.io/projected/f33bb527-27e4-4724-85cc-3a4d0447eaed-kube-api-access-q6dgj\") pod \"f33bb527-27e4-4724-85cc-3a4d0447eaed\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") "
Jul 15 05:47:56.242557 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.242339    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-dshm\") pod \"f33bb527-27e4-4724-85cc-3a4d0447eaed\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") "
Jul 15 05:47:56.242557 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.242445    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f33bb527-27e4-4724-85cc-3a4d0447eaed-tls-certs\") pod \"f33bb527-27e4-4724-85cc-3a4d0447eaed\" (UID: \"f33bb527-27e4-4724-85cc-3a4d0447eaed\") "
Jul 15 05:47:56.242557 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.242448    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-home" (OuterVolumeSpecName: "home") pod "f33bb527-27e4-4724-85cc-3a4d0447eaed" (UID: "f33bb527-27e4-4724-85cc-3a4d0447eaed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:47:56.242835 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.242667    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-home\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:47:56.242897 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.242831    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f33bb527-27e4-4724-85cc-3a4d0447eaed" (UID: "f33bb527-27e4-4724-85cc-3a4d0447eaed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:47:56.243363 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.243334    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-model-cache" (OuterVolumeSpecName: "model-cache") pod "f33bb527-27e4-4724-85cc-3a4d0447eaed" (UID: "f33bb527-27e4-4724-85cc-3a4d0447eaed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:47:56.244773 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.244743    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f33bb527-27e4-4724-85cc-3a4d0447eaed-kube-api-access-q6dgj" (OuterVolumeSpecName: "kube-api-access-q6dgj") pod "f33bb527-27e4-4724-85cc-3a4d0447eaed" (UID: "f33bb527-27e4-4724-85cc-3a4d0447eaed"). InnerVolumeSpecName "kube-api-access-q6dgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:47:56.244773 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.244766    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f33bb527-27e4-4724-85cc-3a4d0447eaed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f33bb527-27e4-4724-85cc-3a4d0447eaed" (UID: "f33bb527-27e4-4724-85cc-3a4d0447eaed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:47:56.244931 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.244808    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-dshm" (OuterVolumeSpecName: "dshm") pod "f33bb527-27e4-4724-85cc-3a4d0447eaed" (UID: "f33bb527-27e4-4724-85cc-3a4d0447eaed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:47:56.337822 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.337782    2562 generic.go:356] "Generic (PLEG): container finished" podID="f33bb527-27e4-4724-85cc-3a4d0447eaed" containerID="b37837593adf4994a65e86a045af39ce151796a20d8fab2ab37d3eb38f551b44" exitCode=0
Jul 15 05:47:56.338027 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.337847    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"
Jul 15 05:47:56.338027 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.337862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8" event={"ID":"f33bb527-27e4-4724-85cc-3a4d0447eaed","Type":"ContainerDied","Data":"b37837593adf4994a65e86a045af39ce151796a20d8fab2ab37d3eb38f551b44"}
Jul 15 05:47:56.338027 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.337904    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8" event={"ID":"f33bb527-27e4-4724-85cc-3a4d0447eaed","Type":"ContainerDied","Data":"7154ccea8bc7a74e292a4e348eebaddceb556d1591bee46d21fa80f0e2abb3bc"}
Jul 15 05:47:56.338027 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.337921    2562 scope.go:117] "RemoveContainer" containerID="b37837593adf4994a65e86a045af39ce151796a20d8fab2ab37d3eb38f551b44"
Jul 15 05:47:56.343428 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.343400    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-model-cache\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:47:56.343570 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.343449    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-tmp-dir\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:47:56.343570 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.343463    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6dgj\" (UniqueName: \"kubernetes.io/projected/f33bb527-27e4-4724-85cc-3a4d0447eaed-kube-api-access-q6dgj\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:47:56.343570 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.343474    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f33bb527-27e4-4724-85cc-3a4d0447eaed-dshm\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:47:56.343570 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.343484    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f33bb527-27e4-4724-85cc-3a4d0447eaed-tls-certs\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:47:56.347507 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.347482    2562 scope.go:117] "RemoveContainer" containerID="b37837593adf4994a65e86a045af39ce151796a20d8fab2ab37d3eb38f551b44"
Jul 15 05:47:56.347804 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:47:56.347787    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b37837593adf4994a65e86a045af39ce151796a20d8fab2ab37d3eb38f551b44\": container with ID starting with b37837593adf4994a65e86a045af39ce151796a20d8fab2ab37d3eb38f551b44 not found: ID does not exist" containerID="b37837593adf4994a65e86a045af39ce151796a20d8fab2ab37d3eb38f551b44"
Jul 15 05:47:56.347857 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.347812    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b37837593adf4994a65e86a045af39ce151796a20d8fab2ab37d3eb38f551b44"} err="failed to get container status \"b37837593adf4994a65e86a045af39ce151796a20d8fab2ab37d3eb38f551b44\": rpc error: code = NotFound desc = could not find container \"b37837593adf4994a65e86a045af39ce151796a20d8fab2ab37d3eb38f551b44\": container with ID starting with b37837593adf4994a65e86a045af39ce151796a20d8fab2ab37d3eb38f551b44 not found: ID does not exist"
Jul 15 05:47:56.360686 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.360630    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"]
Jul 15 05:47:56.362915 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:56.362887    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/sub-test-model-1edf20-kserve-764f45fc55-sqqs8"]
Jul 15 05:47:57.844823 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:47:57.844791    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f33bb527-27e4-4724-85cc-3a4d0447eaed" path="/var/lib/kubelet/pods/f33bb527-27e4-4724-85cc-3a4d0447eaed/volumes"
Jul 15 05:48:02.935525 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:02.935444    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"]
Jul 15 05:48:02.935871 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:02.935770    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f33bb527-27e4-4724-85cc-3a4d0447eaed" containerName="main"
Jul 15 05:48:02.935871 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:02.935781    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f33bb527-27e4-4724-85cc-3a4d0447eaed" containerName="main"
Jul 15 05:48:02.935871 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:02.935831    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f33bb527-27e4-4724-85cc-3a4d0447eaed" containerName="main"
Jul 15 05:48:02.940101 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:02.940085    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:02.943541 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:02.943512    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d43b4e\"/\"rate-test-model-d43b4e-kserve-self-signed-certs\""
Jul 15 05:48:02.943683 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:02.943543    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d43b4e\"/\"kube-root-ca.crt\""
Jul 15 05:48:02.943683 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:02.943554    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d43b4e\"/\"default-dockercfg-h2rd9\""
Jul 15 05:48:02.943683 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:02.943603    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d43b4e\"/\"openshift-service-ca.crt\""
Jul 15 05:48:02.949321 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:02.948402    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"]
Jul 15 05:48:02.952590 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:02.952560    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:48:03.002382 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.002326    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-tls-certs\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.002546 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.002450    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-tmp-dir\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.002613 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.002589    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjqb5\" (UniqueName: \"kubernetes.io/projected/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-kube-api-access-jjqb5\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.002666 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.002652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-dshm\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.002702 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.002676    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-model-cache\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.002702 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.002697    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-home\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.103551 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.103511    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-tls-certs\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.103729 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.103571    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-tmp-dir\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.103729 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.103618    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjqb5\" (UniqueName: \"kubernetes.io/projected/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-kube-api-access-jjqb5\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.103729 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.103670    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-dshm\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.103729 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.103694    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-model-cache\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.103729 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.103727    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-home\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.104064 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.104040    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-tmp-dir\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.104129 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.104088    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-home\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.104178 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.104131    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-model-cache\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.105970 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.105951    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-dshm\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.106342 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.106322    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-tls-certs\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.111437 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.111413    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjqb5\" (UniqueName: \"kubernetes.io/projected/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-kube-api-access-jjqb5\") pod \"rate-test-model-d43b4e-kserve-65979d8b99-9288w\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") " pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.250472 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.250433    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:03.379485 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:03.379461    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"]
Jul 15 05:48:03.382396 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:48:03.382350    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdaead99_15eb_4b0b_84ad_efcd6b17cf23.slice/crio-6a8677c15a26b898284616b0f8fc53bde58fe5c128c5baa7166b80c9f57c7257 WatchSource:0}: Error finding container 6a8677c15a26b898284616b0f8fc53bde58fe5c128c5baa7166b80c9f57c7257: Status 404 returned error can't find the container with id 6a8677c15a26b898284616b0f8fc53bde58fe5c128c5baa7166b80c9f57c7257
Jul 15 05:48:04.375088 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:04.375047    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w" event={"ID":"fdaead99-15eb-4b0b-84ad-efcd6b17cf23","Type":"ContainerStarted","Data":"7cee34b05428beb99af819cdaf7a0ac0c079351282757e61c6539f04e0516eb6"}
Jul 15 05:48:04.375508 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:04.375097    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w" event={"ID":"fdaead99-15eb-4b0b-84ad-efcd6b17cf23","Type":"ContainerStarted","Data":"6a8677c15a26b898284616b0f8fc53bde58fe5c128c5baa7166b80c9f57c7257"}
Jul 15 05:48:04.391983 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:04.391928    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w" podStartSLOduration=2.391911646 podStartE2EDuration="2.391911646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:48:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:48:04.391655278 +0000 UTC m=+2966.995207888" watchObservedRunningTime="2026-07-15 05:48:04.391911646 +0000 UTC m=+2966.995464254"
Jul 15 05:48:13.251315 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:13.251248    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:13.251875 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:13.251331    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:13.263902 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:13.263870    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:13.434596 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:13.434558    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:48:15.536205 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:15.536170    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:48:15.636565 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:15.636533    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:48:23.432220 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:23.432182    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:48:23.532843 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:23.532804    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:48:38.366784 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:38.366626    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:48:38.373984 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:38.373774    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:48:45.239429 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.239389    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"]
Jul 15 05:48:45.242782 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.242764    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.245261 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.245233    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1edf20\"/\"default-dockercfg-mp96r\""
Jul 15 05:48:45.246385 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.246344    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1edf20\"/\"rate-test-model-1edf20-kserve-self-signed-certs\""
Jul 15 05:48:45.246583 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.246568    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1edf20\"/\"kube-root-ca.crt\""
Jul 15 05:48:45.246658 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.246637    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1edf20\"/\"openshift-service-ca.crt\""
Jul 15 05:48:45.250348 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.250321    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"]
Jul 15 05:48:45.274593 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.274555    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f734ad8e-3da1-40a9-af26-f06e2aba6dab-tls-certs\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.274772 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.274613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2jqf\" (UniqueName: \"kubernetes.io/projected/f734ad8e-3da1-40a9-af26-f06e2aba6dab-kube-api-access-h2jqf\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.274772 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.274652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-dshm\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.274772 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.274671    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-model-cache\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.274946 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.274801    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-home\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.274946 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.274856    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-tmp-dir\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.326466 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.326434    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:48:45.375749 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.375711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-home\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.375749 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.375757    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-tmp-dir\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.376021 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.375792    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f734ad8e-3da1-40a9-af26-f06e2aba6dab-tls-certs\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.376021 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.375813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2jqf\" (UniqueName: \"kubernetes.io/projected/f734ad8e-3da1-40a9-af26-f06e2aba6dab-kube-api-access-h2jqf\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.376021 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.375843    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-dshm\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.376021 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.375857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-model-cache\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.376241 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.376145    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-home\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.376290 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.376240    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-model-cache\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.376290 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.376274    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-tmp-dir\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.378155 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.378127    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-dshm\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.378334 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.378314    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f734ad8e-3da1-40a9-af26-f06e2aba6dab-tls-certs\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.383637 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.383615    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2jqf\" (UniqueName: \"kubernetes.io/projected/f734ad8e-3da1-40a9-af26-f06e2aba6dab-kube-api-access-h2jqf\") pod \"rate-test-model-1edf20-kserve-5d69994987-kvpjv\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") " pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.552901 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.552796    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:45.684562 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:45.684525    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"]
Jul 15 05:48:46.577866 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:46.577820    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv" event={"ID":"f734ad8e-3da1-40a9-af26-f06e2aba6dab","Type":"ContainerStarted","Data":"2734f43da2f91617669185b329dfbc8382f95b31b9a2283f60a6c686a23ac7f0"}
Jul 15 05:48:46.577866 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:46.577870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv" event={"ID":"f734ad8e-3da1-40a9-af26-f06e2aba6dab","Type":"ContainerStarted","Data":"eb2379267409f629a2f3cb510b8f981e62e3006741e1b726fbcdeb810560441d"}
Jul 15 05:48:46.597863 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:46.597803    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv" podStartSLOduration=1.5977861469999999 podStartE2EDuration="1.597786147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:48:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:48:46.596874332 +0000 UTC m=+3009.200426951" watchObservedRunningTime="2026-07-15 05:48:46.597786147 +0000 UTC m=+3009.201338756"
Jul 15 05:48:55.553204 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:55.553154    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:55.553204 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:55.553216    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:55.565862 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:55.565832    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:55.629542 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:55.629512    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:48:59.004163 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:48:59.004127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:49:35.466806 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:35.466734    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"]
Jul 15 05:49:35.483463 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:35.483425    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:49:36.453020 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:36.452983    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"]
Jul 15 05:49:36.473146 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:36.473119    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:49:40.536922 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:40.536887    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7590-139709925953072"]
Jul 15 05:49:40.540616 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:40.540591    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709925953072"
Jul 15 05:49:40.542962 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:40.542942    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-wjfck\""
Jul 15 05:49:40.547971 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:40.547914    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7590-139709925953072"]
Jul 15 05:49:40.666825 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:40.666782    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h79m\" (UniqueName: \"kubernetes.io/projected/93ea9241-46ad-4488-bd30-e372434fa829-kube-api-access-2h79m\") pod \"test-curl-7590-139709925953072\" (UID: \"93ea9241-46ad-4488-bd30-e372434fa829\") " pod="openshift-ingress/test-curl-7590-139709925953072"
Jul 15 05:49:40.767228 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:40.767194    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h79m\" (UniqueName: \"kubernetes.io/projected/93ea9241-46ad-4488-bd30-e372434fa829-kube-api-access-2h79m\") pod \"test-curl-7590-139709925953072\" (UID: \"93ea9241-46ad-4488-bd30-e372434fa829\") " pod="openshift-ingress/test-curl-7590-139709925953072"
Jul 15 05:49:40.775761 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:40.775737    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h79m\" (UniqueName: \"kubernetes.io/projected/93ea9241-46ad-4488-bd30-e372434fa829-kube-api-access-2h79m\") pod \"test-curl-7590-139709925953072\" (UID: \"93ea9241-46ad-4488-bd30-e372434fa829\") " pod="openshift-ingress/test-curl-7590-139709925953072"
Jul 15 05:49:40.850218 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:40.850128    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709925953072"
Jul 15 05:49:40.980084 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:40.980061    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7590-139709925953072"]
Jul 15 05:49:40.982554 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:49:40.982523    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93ea9241_46ad_4488_bd30_e372434fa829.slice/crio-db724af89e7911800d73f6f5ad10057de82a144403fe3e9119b4967ecb2f2e34 WatchSource:0}: Error finding container db724af89e7911800d73f6f5ad10057de82a144403fe3e9119b4967ecb2f2e34: Status 404 returned error can't find the container with id db724af89e7911800d73f6f5ad10057de82a144403fe3e9119b4967ecb2f2e34
Jul 15 05:49:40.983867 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:40.983845    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 05:49:41.831586 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:41.831548    2562 generic.go:356] "Generic (PLEG): container finished" podID="93ea9241-46ad-4488-bd30-e372434fa829" containerID="f777d86695225e11873e8570fc02693e86bd7b02ce10525acc900a9ec3278ebe" exitCode=0
Jul 15 05:49:41.831974 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:41.831595    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7590-139709925953072" event={"ID":"93ea9241-46ad-4488-bd30-e372434fa829","Type":"ContainerDied","Data":"f777d86695225e11873e8570fc02693e86bd7b02ce10525acc900a9ec3278ebe"}
Jul 15 05:49:41.831974 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:41.831649    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7590-139709925953072" event={"ID":"93ea9241-46ad-4488-bd30-e372434fa829","Type":"ContainerStarted","Data":"db724af89e7911800d73f6f5ad10057de82a144403fe3e9119b4967ecb2f2e34"}
Jul 15 05:49:42.985030 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:42.984996    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709925953072"
Jul 15 05:49:43.006619 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:43.006590    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7590-139709925953072_93ea9241-46ad-4488-bd30-e372434fa829/test-curl-7590-139709925953072/0.log"
Jul 15 05:49:43.032869 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:43.032833    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7590-139709925953072"]
Jul 15 05:49:43.035134 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:43.035107    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7590-139709925953072"]
Jul 15 05:49:43.090104 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:43.090017    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2h79m\" (UniqueName: \"kubernetes.io/projected/93ea9241-46ad-4488-bd30-e372434fa829-kube-api-access-2h79m\") pod \"93ea9241-46ad-4488-bd30-e372434fa829\" (UID: \"93ea9241-46ad-4488-bd30-e372434fa829\") "
Jul 15 05:49:43.092243 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:43.092210    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ea9241-46ad-4488-bd30-e372434fa829-kube-api-access-2h79m" (OuterVolumeSpecName: "kube-api-access-2h79m") pod "93ea9241-46ad-4488-bd30-e372434fa829" (UID: "93ea9241-46ad-4488-bd30-e372434fa829"). InnerVolumeSpecName "kube-api-access-2h79m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:49:43.191013 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:43.190979    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2h79m\" (UniqueName: \"kubernetes.io/projected/93ea9241-46ad-4488-bd30-e372434fa829-kube-api-access-2h79m\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:43.842703 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:43.842677    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709925953072"
Jul 15 05:49:43.846218 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:43.846191    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93ea9241-46ad-4488-bd30-e372434fa829" path="/var/lib/kubelet/pods/93ea9241-46ad-4488-bd30-e372434fa829/volumes"
Jul 15 05:49:43.846575 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:43.846561    2562 scope.go:117] "RemoveContainer" containerID="f777d86695225e11873e8570fc02693e86bd7b02ce10525acc900a9ec3278ebe"
Jul 15 05:49:46.797688 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:46.797645    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7590-139709924323520"]
Jul 15 05:49:46.798146 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:46.798119    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93ea9241-46ad-4488-bd30-e372434fa829" containerName="test-curl-7590-139709925953072"
Jul 15 05:49:46.798146 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:46.798137    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ea9241-46ad-4488-bd30-e372434fa829" containerName="test-curl-7590-139709925953072"
Jul 15 05:49:46.798276 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:46.798240    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="93ea9241-46ad-4488-bd30-e372434fa829" containerName="test-curl-7590-139709925953072"
Jul 15 05:49:46.802636 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:46.802614    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:46.805482 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:46.805457    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-wjfck\""
Jul 15 05:49:46.806488 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:46.806465    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7590-139709924323520"]
Jul 15 05:49:46.928238 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:46.928201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7chps\" (UniqueName: \"kubernetes.io/projected/e14b205a-7e99-4dac-aaa9-2849dad28611-kube-api-access-7chps\") pod \"test-curl-7590-139709924323520\" (UID: \"e14b205a-7e99-4dac-aaa9-2849dad28611\") " pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:47.028795 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:47.028758    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7chps\" (UniqueName: \"kubernetes.io/projected/e14b205a-7e99-4dac-aaa9-2849dad28611-kube-api-access-7chps\") pod \"test-curl-7590-139709924323520\" (UID: \"e14b205a-7e99-4dac-aaa9-2849dad28611\") " pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:47.037905 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:47.037871    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7chps\" (UniqueName: \"kubernetes.io/projected/e14b205a-7e99-4dac-aaa9-2849dad28611-kube-api-access-7chps\") pod \"test-curl-7590-139709924323520\" (UID: \"e14b205a-7e99-4dac-aaa9-2849dad28611\") " pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:47.112638 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:47.112552    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:47.243916 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:47.243892    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7590-139709924323520"]
Jul 15 05:49:47.245778 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:49:47.245749    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode14b205a_7e99_4dac_aaa9_2849dad28611.slice/crio-dd0f11dc05faaac37626dd246a6b934fea07ecf56ff3f7355087d1842c62beb2 WatchSource:0}: Error finding container dd0f11dc05faaac37626dd246a6b934fea07ecf56ff3f7355087d1842c62beb2: Status 404 returned error can't find the container with id dd0f11dc05faaac37626dd246a6b934fea07ecf56ff3f7355087d1842c62beb2
Jul 15 05:49:47.863710 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:47.863619    2562 generic.go:356] "Generic (PLEG): container finished" podID="e14b205a-7e99-4dac-aaa9-2849dad28611" containerID="bcd1859c1c2df37b3a8867c8a08d2fd8c2079eefb493d9d4050c2041c23973c9" exitCode=0
Jul 15 05:49:47.863710 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:47.863667    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7590-139709924323520" event={"ID":"e14b205a-7e99-4dac-aaa9-2849dad28611","Type":"ContainerDied","Data":"bcd1859c1c2df37b3a8867c8a08d2fd8c2079eefb493d9d4050c2041c23973c9"}
Jul 15 05:49:47.863710 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:47.863692    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7590-139709924323520" event={"ID":"e14b205a-7e99-4dac-aaa9-2849dad28611","Type":"ContainerStarted","Data":"dd0f11dc05faaac37626dd246a6b934fea07ecf56ff3f7355087d1842c62beb2"}
Jul 15 05:49:49.006514 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:49.006483    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:49.025138 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:49.025106    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7590-139709924323520_e14b205a-7e99-4dac-aaa9-2849dad28611/test-curl-7590-139709924323520/0.log"
Jul 15 05:49:49.051579 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:49.051548    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7590-139709924323520"]
Jul 15 05:49:49.055241 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:49.055215    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7590-139709924323520"]
Jul 15 05:49:49.150419 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:49.150314    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7chps\" (UniqueName: \"kubernetes.io/projected/e14b205a-7e99-4dac-aaa9-2849dad28611-kube-api-access-7chps\") pod \"e14b205a-7e99-4dac-aaa9-2849dad28611\" (UID: \"e14b205a-7e99-4dac-aaa9-2849dad28611\") "
Jul 15 05:49:49.152486 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:49.152457    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e14b205a-7e99-4dac-aaa9-2849dad28611-kube-api-access-7chps" (OuterVolumeSpecName: "kube-api-access-7chps") pod "e14b205a-7e99-4dac-aaa9-2849dad28611" (UID: "e14b205a-7e99-4dac-aaa9-2849dad28611"). InnerVolumeSpecName "kube-api-access-7chps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:49:49.252071 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:49.252035    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7chps\" (UniqueName: \"kubernetes.io/projected/e14b205a-7e99-4dac-aaa9-2849dad28611-kube-api-access-7chps\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:49.845544 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:49.845512    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e14b205a-7e99-4dac-aaa9-2849dad28611" path="/var/lib/kubelet/pods/e14b205a-7e99-4dac-aaa9-2849dad28611/volumes"
Jul 15 05:49:49.874033 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:49.874005    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709924323520"
Jul 15 05:49:49.874033 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:49.874025    2562 scope.go:117] "RemoveContainer" containerID="bcd1859c1c2df37b3a8867c8a08d2fd8c2079eefb493d9d4050c2041c23973c9"
Jul 15 05:49:50.486780 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.486735    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w" podUID="fdaead99-15eb-4b0b-84ad-efcd6b17cf23" containerName="main" containerID="cri-o://7cee34b05428beb99af819cdaf7a0ac0c079351282757e61c6539f04e0516eb6" gracePeriod=45
Jul 15 05:49:50.749414 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.749333    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:49:50.866841 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.866805    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjqb5\" (UniqueName: \"kubernetes.io/projected/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-kube-api-access-jjqb5\") pod \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") "
Jul 15 05:49:50.866841 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.866848    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-dshm\") pod \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") "
Jul 15 05:49:50.867101 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.866888    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-tls-certs\") pod \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") "
Jul 15 05:49:50.867101 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.866922    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-home\") pod \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") "
Jul 15 05:49:50.867101 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.866948    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-model-cache\") pod \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") "
Jul 15 05:49:50.867101 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.867001    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-tmp-dir\") pod \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\" (UID: \"fdaead99-15eb-4b0b-84ad-efcd6b17cf23\") "
Jul 15 05:49:50.867310 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.867225    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-home" (OuterVolumeSpecName: "home") pod "fdaead99-15eb-4b0b-84ad-efcd6b17cf23" (UID: "fdaead99-15eb-4b0b-84ad-efcd6b17cf23"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:49:50.867310 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.867252    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-model-cache" (OuterVolumeSpecName: "model-cache") pod "fdaead99-15eb-4b0b-84ad-efcd6b17cf23" (UID: "fdaead99-15eb-4b0b-84ad-efcd6b17cf23"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:49:50.867434 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.867329    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-home\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:50.867434 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.867351    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fdaead99-15eb-4b0b-84ad-efcd6b17cf23" (UID: "fdaead99-15eb-4b0b-84ad-efcd6b17cf23"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:49:50.869142 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.869120    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-dshm" (OuterVolumeSpecName: "dshm") pod "fdaead99-15eb-4b0b-84ad-efcd6b17cf23" (UID: "fdaead99-15eb-4b0b-84ad-efcd6b17cf23"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:49:50.869142 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.869121    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-kube-api-access-jjqb5" (OuterVolumeSpecName: "kube-api-access-jjqb5") pod "fdaead99-15eb-4b0b-84ad-efcd6b17cf23" (UID: "fdaead99-15eb-4b0b-84ad-efcd6b17cf23"). InnerVolumeSpecName "kube-api-access-jjqb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:49:50.869142 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.869142    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fdaead99-15eb-4b0b-84ad-efcd6b17cf23" (UID: "fdaead99-15eb-4b0b-84ad-efcd6b17cf23"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:49:50.879881 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.879851    2562 generic.go:356] "Generic (PLEG): container finished" podID="fdaead99-15eb-4b0b-84ad-efcd6b17cf23" containerID="7cee34b05428beb99af819cdaf7a0ac0c079351282757e61c6539f04e0516eb6" exitCode=0
Jul 15 05:49:50.880007 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.879895    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"
Jul 15 05:49:50.880007 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.879939    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w" event={"ID":"fdaead99-15eb-4b0b-84ad-efcd6b17cf23","Type":"ContainerDied","Data":"7cee34b05428beb99af819cdaf7a0ac0c079351282757e61c6539f04e0516eb6"}
Jul 15 05:49:50.880007 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.879993    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w" event={"ID":"fdaead99-15eb-4b0b-84ad-efcd6b17cf23","Type":"ContainerDied","Data":"6a8677c15a26b898284616b0f8fc53bde58fe5c128c5baa7166b80c9f57c7257"}
Jul 15 05:49:50.880202 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.880008    2562 scope.go:117] "RemoveContainer" containerID="7cee34b05428beb99af819cdaf7a0ac0c079351282757e61c6539f04e0516eb6"
Jul 15 05:49:50.888902 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.888887    2562 scope.go:117] "RemoveContainer" containerID="7cee34b05428beb99af819cdaf7a0ac0c079351282757e61c6539f04e0516eb6"
Jul 15 05:49:50.889153 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:49:50.889135    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cee34b05428beb99af819cdaf7a0ac0c079351282757e61c6539f04e0516eb6\": container with ID starting with 7cee34b05428beb99af819cdaf7a0ac0c079351282757e61c6539f04e0516eb6 not found: ID does not exist" containerID="7cee34b05428beb99af819cdaf7a0ac0c079351282757e61c6539f04e0516eb6"
Jul 15 05:49:50.889217 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.889162    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cee34b05428beb99af819cdaf7a0ac0c079351282757e61c6539f04e0516eb6"} err="failed to get container status \"7cee34b05428beb99af819cdaf7a0ac0c079351282757e61c6539f04e0516eb6\": rpc error: code = NotFound desc = could not find container \"7cee34b05428beb99af819cdaf7a0ac0c079351282757e61c6539f04e0516eb6\": container with ID starting with 7cee34b05428beb99af819cdaf7a0ac0c079351282757e61c6539f04e0516eb6 not found: ID does not exist"
Jul 15 05:49:50.900212 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.900186    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"]
Jul 15 05:49:50.904080 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.904060    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d43b4e/rate-test-model-d43b4e-kserve-65979d8b99-9288w"]
Jul 15 05:49:50.968024 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.967989    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-tmp-dir\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:50.968024 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.968018    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjqb5\" (UniqueName: \"kubernetes.io/projected/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-kube-api-access-jjqb5\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:50.968024 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.968027    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-dshm\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:50.968267 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.968039    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-tls-certs\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:50.968267 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:50.968049    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdaead99-15eb-4b0b-84ad-efcd6b17cf23-model-cache\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:51.473179 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.473133    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv" podUID="f734ad8e-3da1-40a9-af26-f06e2aba6dab" containerName="main" containerID="cri-o://2734f43da2f91617669185b329dfbc8382f95b31b9a2283f60a6c686a23ac7f0" gracePeriod=45
Jul 15 05:49:51.724997 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.724938    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:49:51.800226 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.800195    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7590-139709926249792"]
Jul 15 05:49:51.800562 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.800548    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdaead99-15eb-4b0b-84ad-efcd6b17cf23" containerName="main"
Jul 15 05:49:51.800608 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.800564    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdaead99-15eb-4b0b-84ad-efcd6b17cf23" containerName="main"
Jul 15 05:49:51.800608 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.800586    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e14b205a-7e99-4dac-aaa9-2849dad28611" containerName="test-curl-7590-139709924323520"
Jul 15 05:49:51.800608 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.800595    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14b205a-7e99-4dac-aaa9-2849dad28611" containerName="test-curl-7590-139709924323520"
Jul 15 05:49:51.800608 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.800602    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f734ad8e-3da1-40a9-af26-f06e2aba6dab" containerName="main"
Jul 15 05:49:51.800608 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.800608    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f734ad8e-3da1-40a9-af26-f06e2aba6dab" containerName="main"
Jul 15 05:49:51.800759 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.800668    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f734ad8e-3da1-40a9-af26-f06e2aba6dab" containerName="main"
Jul 15 05:49:51.800759 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.800675    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e14b205a-7e99-4dac-aaa9-2849dad28611" containerName="test-curl-7590-139709924323520"
Jul 15 05:49:51.800759 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.800684    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdaead99-15eb-4b0b-84ad-efcd6b17cf23" containerName="main"
Jul 15 05:49:51.803496 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.803478    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:51.806201 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.806149    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-wjfck\""
Jul 15 05:49:51.808433 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.808413    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7590-139709926249792"]
Jul 15 05:49:51.844976 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.844946    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdaead99-15eb-4b0b-84ad-efcd6b17cf23" path="/var/lib/kubelet/pods/fdaead99-15eb-4b0b-84ad-efcd6b17cf23/volumes"
Jul 15 05:49:51.876941 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.876912    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-model-cache\") pod \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") "
Jul 15 05:49:51.877102 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.876950    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-tmp-dir\") pod \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") "
Jul 15 05:49:51.877102 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.876972    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f734ad8e-3da1-40a9-af26-f06e2aba6dab-tls-certs\") pod \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") "
Jul 15 05:49:51.877102 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.876990    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-dshm\") pod \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") "
Jul 15 05:49:51.877102 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.877086    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2jqf\" (UniqueName: \"kubernetes.io/projected/f734ad8e-3da1-40a9-af26-f06e2aba6dab-kube-api-access-h2jqf\") pod \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") "
Jul 15 05:49:51.877301 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.877156    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-home\") pod \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\" (UID: \"f734ad8e-3da1-40a9-af26-f06e2aba6dab\") "
Jul 15 05:49:51.877301 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.877198    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-model-cache" (OuterVolumeSpecName: "model-cache") pod "f734ad8e-3da1-40a9-af26-f06e2aba6dab" (UID: "f734ad8e-3da1-40a9-af26-f06e2aba6dab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:49:51.877301 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.877213    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f734ad8e-3da1-40a9-af26-f06e2aba6dab" (UID: "f734ad8e-3da1-40a9-af26-f06e2aba6dab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:49:51.877510 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.877314    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5fjw\" (UniqueName: \"kubernetes.io/projected/8cf18990-ee78-4597-a3b1-ddd4cce25d1a-kube-api-access-p5fjw\") pod \"test-curl-7590-139709926249792\" (UID: \"8cf18990-ee78-4597-a3b1-ddd4cce25d1a\") " pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:51.877510 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.877419    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-home" (OuterVolumeSpecName: "home") pod "f734ad8e-3da1-40a9-af26-f06e2aba6dab" (UID: "f734ad8e-3da1-40a9-af26-f06e2aba6dab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:49:51.877510 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.877509    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-home\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:51.877657 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.877523    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-model-cache\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:51.877657 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.877539    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-tmp-dir\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:51.879200 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.879167    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-dshm" (OuterVolumeSpecName: "dshm") pod "f734ad8e-3da1-40a9-af26-f06e2aba6dab" (UID: "f734ad8e-3da1-40a9-af26-f06e2aba6dab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 05:49:51.879305 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.879207    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f734ad8e-3da1-40a9-af26-f06e2aba6dab-kube-api-access-h2jqf" (OuterVolumeSpecName: "kube-api-access-h2jqf") pod "f734ad8e-3da1-40a9-af26-f06e2aba6dab" (UID: "f734ad8e-3da1-40a9-af26-f06e2aba6dab"). InnerVolumeSpecName "kube-api-access-h2jqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:49:51.879305 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.879254    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f734ad8e-3da1-40a9-af26-f06e2aba6dab-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f734ad8e-3da1-40a9-af26-f06e2aba6dab" (UID: "f734ad8e-3da1-40a9-af26-f06e2aba6dab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 05:49:51.887613 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.887589    2562 generic.go:356] "Generic (PLEG): container finished" podID="f734ad8e-3da1-40a9-af26-f06e2aba6dab" containerID="2734f43da2f91617669185b329dfbc8382f95b31b9a2283f60a6c686a23ac7f0" exitCode=0
Jul 15 05:49:51.887713 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.887641    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"
Jul 15 05:49:51.887713 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.887667    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv" event={"ID":"f734ad8e-3da1-40a9-af26-f06e2aba6dab","Type":"ContainerDied","Data":"2734f43da2f91617669185b329dfbc8382f95b31b9a2283f60a6c686a23ac7f0"}
Jul 15 05:49:51.887713 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.887710    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv" event={"ID":"f734ad8e-3da1-40a9-af26-f06e2aba6dab","Type":"ContainerDied","Data":"eb2379267409f629a2f3cb510b8f981e62e3006741e1b726fbcdeb810560441d"}
Jul 15 05:49:51.887819 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.887727    2562 scope.go:117] "RemoveContainer" containerID="2734f43da2f91617669185b329dfbc8382f95b31b9a2283f60a6c686a23ac7f0"
Jul 15 05:49:51.899172 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.899153    2562 scope.go:117] "RemoveContainer" containerID="2734f43da2f91617669185b329dfbc8382f95b31b9a2283f60a6c686a23ac7f0"
Jul 15 05:49:51.899459 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:49:51.899439    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2734f43da2f91617669185b329dfbc8382f95b31b9a2283f60a6c686a23ac7f0\": container with ID starting with 2734f43da2f91617669185b329dfbc8382f95b31b9a2283f60a6c686a23ac7f0 not found: ID does not exist" containerID="2734f43da2f91617669185b329dfbc8382f95b31b9a2283f60a6c686a23ac7f0"
Jul 15 05:49:51.899523 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.899473    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2734f43da2f91617669185b329dfbc8382f95b31b9a2283f60a6c686a23ac7f0"} err="failed to get container status \"2734f43da2f91617669185b329dfbc8382f95b31b9a2283f60a6c686a23ac7f0\": rpc error: code = NotFound desc = could not find container \"2734f43da2f91617669185b329dfbc8382f95b31b9a2283f60a6c686a23ac7f0\": container with ID starting with 2734f43da2f91617669185b329dfbc8382f95b31b9a2283f60a6c686a23ac7f0 not found: ID does not exist"
Jul 15 05:49:51.911559 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.911535    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"]
Jul 15 05:49:51.916919 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.916898    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1edf20/rate-test-model-1edf20-kserve-5d69994987-kvpjv"]
Jul 15 05:49:51.978689 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.978586    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5fjw\" (UniqueName: \"kubernetes.io/projected/8cf18990-ee78-4597-a3b1-ddd4cce25d1a-kube-api-access-p5fjw\") pod \"test-curl-7590-139709926249792\" (UID: \"8cf18990-ee78-4597-a3b1-ddd4cce25d1a\") " pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:51.978860 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.978700    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f734ad8e-3da1-40a9-af26-f06e2aba6dab-dshm\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:51.978860 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.978718    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2jqf\" (UniqueName: \"kubernetes.io/projected/f734ad8e-3da1-40a9-af26-f06e2aba6dab-kube-api-access-h2jqf\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:51.978860 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.978731    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f734ad8e-3da1-40a9-af26-f06e2aba6dab-tls-certs\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:51.985948 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:51.985917    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5fjw\" (UniqueName: \"kubernetes.io/projected/8cf18990-ee78-4597-a3b1-ddd4cce25d1a-kube-api-access-p5fjw\") pod \"test-curl-7590-139709926249792\" (UID: \"8cf18990-ee78-4597-a3b1-ddd4cce25d1a\") " pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:52.114014 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:52.113975    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:52.240445 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:52.240419    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7590-139709926249792"]
Jul 15 05:49:52.242258 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:49:52.242225    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cf18990_ee78_4597_a3b1_ddd4cce25d1a.slice/crio-90f38758a55af3f9e2671157db36ed1fca69e07cf66a2129f32d042e7c8838f1 WatchSource:0}: Error finding container 90f38758a55af3f9e2671157db36ed1fca69e07cf66a2129f32d042e7c8838f1: Status 404 returned error can't find the container with id 90f38758a55af3f9e2671157db36ed1fca69e07cf66a2129f32d042e7c8838f1
Jul 15 05:49:52.895325 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:52.895237    2562 generic.go:356] "Generic (PLEG): container finished" podID="8cf18990-ee78-4597-a3b1-ddd4cce25d1a" containerID="8fbd6410a9d47eed9e30c708f159d50b55d9f4048545e771ef0515d3c0d7684e" exitCode=0
Jul 15 05:49:52.895325 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:52.895286    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7590-139709926249792" event={"ID":"8cf18990-ee78-4597-a3b1-ddd4cce25d1a","Type":"ContainerDied","Data":"8fbd6410a9d47eed9e30c708f159d50b55d9f4048545e771ef0515d3c0d7684e"}
Jul 15 05:49:52.895325 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:52.895313    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7590-139709926249792" event={"ID":"8cf18990-ee78-4597-a3b1-ddd4cce25d1a","Type":"ContainerStarted","Data":"90f38758a55af3f9e2671157db36ed1fca69e07cf66a2129f32d042e7c8838f1"}
Jul 15 05:49:53.849054 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:53.849020    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f734ad8e-3da1-40a9-af26-f06e2aba6dab" path="/var/lib/kubelet/pods/f734ad8e-3da1-40a9-af26-f06e2aba6dab/volumes"
Jul 15 05:49:54.034252 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.034225    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:54.053884 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.053856    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7590-139709926249792_8cf18990-ee78-4597-a3b1-ddd4cce25d1a/test-curl-7590-139709926249792/0.log"
Jul 15 05:49:54.076921 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.076887    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7590-139709926249792"]
Jul 15 05:49:54.080637 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.080610    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7590-139709926249792"]
Jul 15 05:49:54.199577 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.199549    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5fjw\" (UniqueName: \"kubernetes.io/projected/8cf18990-ee78-4597-a3b1-ddd4cce25d1a-kube-api-access-p5fjw\") pod \"8cf18990-ee78-4597-a3b1-ddd4cce25d1a\" (UID: \"8cf18990-ee78-4597-a3b1-ddd4cce25d1a\") "
Jul 15 05:49:54.201655 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.201633    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf18990-ee78-4597-a3b1-ddd4cce25d1a-kube-api-access-p5fjw" (OuterVolumeSpecName: "kube-api-access-p5fjw") pod "8cf18990-ee78-4597-a3b1-ddd4cce25d1a" (UID: "8cf18990-ee78-4597-a3b1-ddd4cce25d1a"). InnerVolumeSpecName "kube-api-access-p5fjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 05:49:54.279514 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.279480    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r"]
Jul 15 05:49:54.279816 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.279803    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf18990-ee78-4597-a3b1-ddd4cce25d1a" containerName="test-curl-7590-139709926249792"
Jul 15 05:49:54.279874 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.279820    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf18990-ee78-4597-a3b1-ddd4cce25d1a" containerName="test-curl-7590-139709926249792"
Jul 15 05:49:54.279909 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.279882    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf18990-ee78-4597-a3b1-ddd4cce25d1a" containerName="test-curl-7590-139709926249792"
Jul 15 05:49:54.282722 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.282704    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r"
Jul 15 05:49:54.285196 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.285174    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 05:49:54.285303 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.285174    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2zt2\""
Jul 15 05:49:54.289124 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.289102    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r"]
Jul 15 05:49:54.300112 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.300081    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5fjw\" (UniqueName: \"kubernetes.io/projected/8cf18990-ee78-4597-a3b1-ddd4cce25d1a-kube-api-access-p5fjw\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:54.401250 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.401208    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5458d498-9b65-413a-b6be-2083ccb1aa32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r\" (UID: \"5458d498-9b65-413a-b6be-2083ccb1aa32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r"
Jul 15 05:49:54.502265 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.502175    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5458d498-9b65-413a-b6be-2083ccb1aa32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r\" (UID: \"5458d498-9b65-413a-b6be-2083ccb1aa32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r"
Jul 15 05:49:54.502914 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.502893    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5458d498-9b65-413a-b6be-2083ccb1aa32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r\" (UID: \"5458d498-9b65-413a-b6be-2083ccb1aa32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r"
Jul 15 05:49:54.593866 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.593833    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r"
Jul 15 05:49:54.722451 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.722426    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r"]
Jul 15 05:49:54.724514 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:49:54.724487    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5458d498_9b65_413a_b6be_2083ccb1aa32.slice/crio-736a19ee77de09723e8024fcd7797b426f74cb89517ac6099ceb663553430f4e WatchSource:0}: Error finding container 736a19ee77de09723e8024fcd7797b426f74cb89517ac6099ceb663553430f4e: Status 404 returned error can't find the container with id 736a19ee77de09723e8024fcd7797b426f74cb89517ac6099ceb663553430f4e
Jul 15 05:49:54.858341 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.858291    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:49:54.906028 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.906000    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:54.906028 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.906028    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90f38758a55af3f9e2671157db36ed1fca69e07cf66a2129f32d042e7c8838f1"
Jul 15 05:49:54.907534 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.907506    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r" event={"ID":"5458d498-9b65-413a-b6be-2083ccb1aa32","Type":"ContainerStarted","Data":"736a19ee77de09723e8024fcd7797b426f74cb89517ac6099ceb663553430f4e"}
Jul 15 05:49:54.916331 ip-10-0-128-231 kubenswrapper[2562]: E0715 05:49:54.916299    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7590-139709926249792\" is forbidden: User \"system:node:ip-10-0-128-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-231.ec2.internal' and this object" podUID="8cf18990-ee78-4597-a3b1-ddd4cce25d1a" pod="openshift-ingress/test-curl-7590-139709926249792"
Jul 15 05:49:54.964276 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:54.964243    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:49:55.845187 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:55.845103    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cf18990-ee78-4597-a3b1-ddd4cce25d1a" path="/var/lib/kubelet/pods/8cf18990-ee78-4597-a3b1-ddd4cce25d1a/volumes"
Jul 15 05:49:55.913978 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:55.913945    2562 generic.go:356] "Generic (PLEG): container finished" podID="5458d498-9b65-413a-b6be-2083ccb1aa32" containerID="b2ffdf8f8f0c37fddf43b851bd029fa9e1f4405f8be0bbb2978ca3434ebcc687" exitCode=0
Jul 15 05:49:55.914128 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:55.914035    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r" event={"ID":"5458d498-9b65-413a-b6be-2083ccb1aa32","Type":"ContainerDied","Data":"b2ffdf8f8f0c37fddf43b851bd029fa9e1f4405f8be0bbb2978ca3434ebcc687"}
Jul 15 05:49:57.045631 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:57.045607    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r"
Jul 15 05:49:57.126943 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:57.126907    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5458d498-9b65-413a-b6be-2083ccb1aa32-maas-api-service-ca\") pod \"5458d498-9b65-413a-b6be-2083ccb1aa32\" (UID: \"5458d498-9b65-413a-b6be-2083ccb1aa32\") "
Jul 15 05:49:57.127270 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:57.127247    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5458d498-9b65-413a-b6be-2083ccb1aa32-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5458d498-9b65-413a-b6be-2083ccb1aa32" (UID: "5458d498-9b65-413a-b6be-2083ccb1aa32"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:49:57.227539 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:57.227500    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5458d498-9b65-413a-b6be-2083ccb1aa32-maas-api-service-ca\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:49:57.923856 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:57.923825    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r"
Jul 15 05:49:57.923856 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:57.923834    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r" event={"ID":"5458d498-9b65-413a-b6be-2083ccb1aa32","Type":"ContainerDied","Data":"736a19ee77de09723e8024fcd7797b426f74cb89517ac6099ceb663553430f4e"}
Jul 15 05:49:57.924069 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:49:57.923887    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="736a19ee77de09723e8024fcd7797b426f74cb89517ac6099ceb663553430f4e"
Jul 15 05:50:04.704995 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:04.704962    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:04.710225 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:04.710197    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d43b4e-29734905-srb7b"]
Jul 15 05:50:04.712712 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:04.712691    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d43b4e-29734905-srb7b"]
Jul 15 05:50:05.844793 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:05.844759    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e638abc7-e796-4c3e-978b-a783008ee080" path="/var/lib/kubelet/pods/e638abc7-e796-4c3e-978b-a783008ee080/volumes"
Jul 15 05:50:09.381179 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:09.381140    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:15.184685 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:15.184648    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:15.285359 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:15.285315    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:20.090147 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.090110    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:20.351325 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.351243    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:20.467696 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.467663    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc"]
Jul 15 05:50:20.468039 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.468024    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5458d498-9b65-413a-b6be-2083ccb1aa32" containerName="revoke-keys"
Jul 15 05:50:20.468084 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.468043    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5458d498-9b65-413a-b6be-2083ccb1aa32" containerName="revoke-keys"
Jul 15 05:50:20.468123 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.468102    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5458d498-9b65-413a-b6be-2083ccb1aa32" containerName="revoke-keys"
Jul 15 05:50:20.471008 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.470989    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc"
Jul 15 05:50:20.473473 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.473450    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b2zt2\""
Jul 15 05:50:20.473613 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.473454    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 05:50:20.477702 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.477675    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc"]
Jul 15 05:50:20.528678 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.528642    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5203fff-9626-48c0-b7d3-9476a200f4c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc\" (UID: \"e5203fff-9626-48c0-b7d3-9476a200f4c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc"
Jul 15 05:50:20.629277 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.629176    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5203fff-9626-48c0-b7d3-9476a200f4c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc\" (UID: \"e5203fff-9626-48c0-b7d3-9476a200f4c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc"
Jul 15 05:50:20.629759 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.629737    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5203fff-9626-48c0-b7d3-9476a200f4c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc\" (UID: \"e5203fff-9626-48c0-b7d3-9476a200f4c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc"
Jul 15 05:50:20.783109 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.783065    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc"
Jul 15 05:50:20.921555 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:20.921526    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc"]
Jul 15 05:50:20.924183 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:50:20.924156    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5203fff_9626_48c0_b7d3_9476a200f4c2.slice/crio-ebc226a67696aa5044c2e473beaa7738578201aa213cf41bbe488d6329b98c4e WatchSource:0}: Error finding container ebc226a67696aa5044c2e473beaa7738578201aa213cf41bbe488d6329b98c4e: Status 404 returned error can't find the container with id ebc226a67696aa5044c2e473beaa7738578201aa213cf41bbe488d6329b98c4e
Jul 15 05:50:21.037310 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:21.037274    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc" event={"ID":"e5203fff-9626-48c0-b7d3-9476a200f4c2","Type":"ContainerStarted","Data":"ebc226a67696aa5044c2e473beaa7738578201aa213cf41bbe488d6329b98c4e"}
Jul 15 05:50:21.055676 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:21.055644    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:21.152784 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:21.152750    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:22.043795 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:22.043756    2562 generic.go:356] "Generic (PLEG): container finished" podID="e5203fff-9626-48c0-b7d3-9476a200f4c2" containerID="164e90f87aabd5e10c55619672450e5d94a2aeb61e0d7ecfb65297656be6d229" exitCode=0
Jul 15 05:50:22.043970 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:22.043838    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc" event={"ID":"e5203fff-9626-48c0-b7d3-9476a200f4c2","Type":"ContainerDied","Data":"164e90f87aabd5e10c55619672450e5d94a2aeb61e0d7ecfb65297656be6d229"}
Jul 15 05:50:23.177224 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:23.177199    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc"
Jul 15 05:50:23.250564 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:23.250524    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5203fff-9626-48c0-b7d3-9476a200f4c2-maas-api-service-ca\") pod \"e5203fff-9626-48c0-b7d3-9476a200f4c2\" (UID: \"e5203fff-9626-48c0-b7d3-9476a200f4c2\") "
Jul 15 05:50:23.250901 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:23.250878    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5203fff-9626-48c0-b7d3-9476a200f4c2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e5203fff-9626-48c0-b7d3-9476a200f4c2" (UID: "e5203fff-9626-48c0-b7d3-9476a200f4c2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 05:50:23.351636 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:23.351540    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5203fff-9626-48c0-b7d3-9476a200f4c2-maas-api-service-ca\") on node \"ip-10-0-128-231.ec2.internal\" DevicePath \"\""
Jul 15 05:50:24.054105 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:24.054075    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc"
Jul 15 05:50:24.054278 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:24.054121    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc" event={"ID":"e5203fff-9626-48c0-b7d3-9476a200f4c2","Type":"ContainerDied","Data":"ebc226a67696aa5044c2e473beaa7738578201aa213cf41bbe488d6329b98c4e"}
Jul 15 05:50:24.054278 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:24.054166    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebc226a67696aa5044c2e473beaa7738578201aa213cf41bbe488d6329b98c4e"
Jul 15 05:50:30.887234 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:30.887196    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1edf20-29734905-6gx2n"]
Jul 15 05:50:30.892092 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:30.892060    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:30.892430 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:30.892411    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1edf20-29734905-6gx2n"]
Jul 15 05:50:31.844582 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:31.844550    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c85822b6-3f60-4f5e-8dc6-54706e8db05d" path="/var/lib/kubelet/pods/c85822b6-3f60-4f5e-8dc6-54706e8db05d/volumes"
Jul 15 05:50:35.560743 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:35.560703    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:41.407871 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:41.407836    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:41.461900 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:41.461869    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:46.272884 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:46.272850    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:46.517089 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:46.517053    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:50:52.895062 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:52.895029    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749c75689b-sqgqr_1780b906-079c-414c-aa53-aea05c06bbbf/copy-manifests/0.log"
Jul 15 05:50:52.901579 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:52.901546    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749c75689b-sqgqr_1780b906-079c-414c-aa53-aea05c06bbbf/manager/0.log"
Jul 15 05:50:53.623584 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:53.623550    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734905-kvlbj_1f7a50bf-e87d-4261-9e08-739cd95560ef/cleanup/0.log"
Jul 15 05:50:53.971036 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:53.971005    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-ff68b6-vglvj_cf7892a2-4fab-44f4-a2af-6ab6509ee758/revoke-keys/0.log"
Jul 15 05:50:54.088404 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:54.088361    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-870c8a-g5xwq_13c0d7fd-76c3-42bb-8438-97bddc5ecb81/revoke-keys/0.log"
Jul 15 05:50:54.905120 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:54.905093    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-d43b4e-8289r_5458d498-9b65-413a-b6be-2083ccb1aa32/revoke-keys/0.log"
Jul 15 05:50:55.023920 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:55.023888    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-1edf20-bhwbc_e5203fff-9626-48c0-b7d3-9476a200f4c2/revoke-keys/0.log"
Jul 15 05:50:56.688615 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:56.688586    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7bmpd_7e748288-9d39-4c24-b5de-717d2362edfc/manager/0.log"
Jul 15 05:50:56.805449 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:56.805420    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-n8q2w_922807c7-647b-4b4c-af54-d88b1f65b3f4/kuadrant-console-plugin/0.log"
Jul 15 05:50:57.290992 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:57.290961    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-k27q4_3f17e5dd-a08b-41a7-8032-cdec4fcc2ddb/manager/0.log"
Jul 15 05:50:57.916190 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:57.916158    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57bf8bf567-pzr2h_0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61/kube-auth-proxy/0.log"
Jul 15 05:50:58.863806 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:58.863777    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-w8xzq_fda1dee7-5826-4fa5-96be-e25fc312e74a/main/0.log"
Jul 15 05:50:59.101611 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:59.101578    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-lsk7h_c88f43e2-8be8-401c-a146-ccf40c1b06ed/main/0.log"
Jul 15 05:50:59.454401 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:50:59.454346    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-n449z_c2099183-1f82-4dc7-88de-519cf3097d75/main/0.log"
Jul 15 05:51:04.037805 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.037771    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bw8rh/must-gather-b5gc9"]
Jul 15 05:51:04.038403 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.038250    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5203fff-9626-48c0-b7d3-9476a200f4c2" containerName="revoke-keys"
Jul 15 05:51:04.038403 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.038270    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5203fff-9626-48c0-b7d3-9476a200f4c2" containerName="revoke-keys"
Jul 15 05:51:04.038403 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.038395    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5203fff-9626-48c0-b7d3-9476a200f4c2" containerName="revoke-keys"
Jul 15 05:51:04.041495 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.041471    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:51:04.041660 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.041480    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bw8rh/must-gather-b5gc9"
Jul 15 05:51:04.047805 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.047781    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bw8rh\"/\"default-dockercfg-b5756\""
Jul 15 05:51:04.048683 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.048656    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bw8rh\"/\"kube-root-ca.crt\""
Jul 15 05:51:04.048817 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.048709    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bw8rh\"/\"openshift-service-ca.crt\""
Jul 15 05:51:04.051230 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.051202    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bw8rh/must-gather-b5gc9"]
Jul 15 05:51:04.096742 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.096712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log" limit=1024
Jul 15 05:51:04.110014 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.109984    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfr9t\" (UniqueName: \"kubernetes.io/projected/38050c51-5e58-4457-8ce3-1f2afda4a3ef-kube-api-access-gfr9t\") pod \"must-gather-b5gc9\" (UID: \"38050c51-5e58-4457-8ce3-1f2afda4a3ef\") " pod="openshift-must-gather-bw8rh/must-gather-b5gc9"
Jul 15 05:51:04.110153 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.110056    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/38050c51-5e58-4457-8ce3-1f2afda4a3ef-must-gather-output\") pod \"must-gather-b5gc9\" (UID: \"38050c51-5e58-4457-8ce3-1f2afda4a3ef\") " pod="openshift-must-gather-bw8rh/must-gather-b5gc9"
Jul 15 05:51:04.210634 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.210601    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfr9t\" (UniqueName: \"kubernetes.io/projected/38050c51-5e58-4457-8ce3-1f2afda4a3ef-kube-api-access-gfr9t\") pod \"must-gather-b5gc9\" (UID: \"38050c51-5e58-4457-8ce3-1f2afda4a3ef\") " pod="openshift-must-gather-bw8rh/must-gather-b5gc9"
Jul 15 05:51:04.210814 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.210660    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/38050c51-5e58-4457-8ce3-1f2afda4a3ef-must-gather-output\") pod \"must-gather-b5gc9\" (UID: \"38050c51-5e58-4457-8ce3-1f2afda4a3ef\") " pod="openshift-must-gather-bw8rh/must-gather-b5gc9"
Jul 15 05:51:04.210952 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.210935    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/38050c51-5e58-4457-8ce3-1f2afda4a3ef-must-gather-output\") pod \"must-gather-b5gc9\" (UID: \"38050c51-5e58-4457-8ce3-1f2afda4a3ef\") " pod="openshift-must-gather-bw8rh/must-gather-b5gc9"
Jul 15 05:51:04.225107 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.225074    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfr9t\" (UniqueName: \"kubernetes.io/projected/38050c51-5e58-4457-8ce3-1f2afda4a3ef-kube-api-access-gfr9t\") pod \"must-gather-b5gc9\" (UID: \"38050c51-5e58-4457-8ce3-1f2afda4a3ef\") " pod="openshift-must-gather-bw8rh/must-gather-b5gc9"
Jul 15 05:51:04.357390 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.357269    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bw8rh/must-gather-b5gc9"
Jul 15 05:51:04.491603 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:04.491571    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bw8rh/must-gather-b5gc9"]
Jul 15 05:51:04.493591 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:51:04.493560    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38050c51_5e58_4457_8ce3_1f2afda4a3ef.slice/crio-d7ac91856b0213b6bca09febbc43702b9874f3ef794c68495acbabcf873a666d WatchSource:0}: Error finding container d7ac91856b0213b6bca09febbc43702b9874f3ef794c68495acbabcf873a666d: Status 404 returned error can't find the container with id d7ac91856b0213b6bca09febbc43702b9874f3ef794c68495acbabcf873a666d
Jul 15 05:51:05.247906 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:05.247870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bw8rh/must-gather-b5gc9" event={"ID":"38050c51-5e58-4457-8ce3-1f2afda4a3ef","Type":"ContainerStarted","Data":"d7ac91856b0213b6bca09febbc43702b9874f3ef794c68495acbabcf873a666d"}
Jul 15 05:51:06.256732 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:06.256691    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bw8rh/must-gather-b5gc9" event={"ID":"38050c51-5e58-4457-8ce3-1f2afda4a3ef","Type":"ContainerStarted","Data":"db18b33e60358b51162e45d6214611ea0fba7985eb057805211981747b0dba37"}
Jul 15 05:51:06.257135 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:06.256746    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bw8rh/must-gather-b5gc9" event={"ID":"38050c51-5e58-4457-8ce3-1f2afda4a3ef","Type":"ContainerStarted","Data":"11da058daf3159b6f55fc44682c4e118d118f71bb1ecac9d56d6f371388ef29d"}
Jul 15 05:51:06.272209 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:06.272138    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bw8rh/must-gather-b5gc9" podStartSLOduration=1.3823496579999999 podStartE2EDuration="2.272115575s" totalImagesPullingTime="889.765917ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 05:51:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:51:06.271028219 +0000 UTC m=+3148.874580829" watchObservedRunningTime="2026-07-15 05:51:06.272115575 +0000 UTC m=+3148.875668185"
Jul 15 05:51:06.943947 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:06.943917    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tkz8s_7dced01d-9de8-4e21-8ca3-241d08ecab2e/global-pull-secret-syncer/0.log"
Jul 15 05:51:07.031608 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:07.031574    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gp2nd_e49b01bc-0081-4961-90a6-0b2fd594abf7/konnectivity-agent/0.log"
Jul 15 05:51:07.129924 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:07.129897    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-231.ec2.internal_33e99f318c4c76ca78a270f6e363c8e1/haproxy/0.log"
Jul 15 05:51:11.374480 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:11.374432    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7bmpd_7e748288-9d39-4c24-b5de-717d2362edfc/manager/0.log"
Jul 15 05:51:11.411150 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:11.411122    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-n8q2w_922807c7-647b-4b4c-af54-d88b1f65b3f4/kuadrant-console-plugin/0.log"
Jul 15 05:51:11.677443 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:11.674924    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-k27q4_3f17e5dd-a08b-41a7-8032-cdec4fcc2ddb/manager/0.log"
Jul 15 05:51:13.663641 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:13.663609    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-xwctc_6169dacf-0931-4554-b48e-5da0d8b36ddd/cluster-monitoring-operator/0.log"
Jul 15 05:51:13.966484 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:13.966394    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n88cb_8c760c2f-4a4b-4d8d-990f-325ad5eb6846/node-exporter/0.log"
Jul 15 05:51:13.991034 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:13.990957    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n88cb_8c760c2f-4a4b-4d8d-990f-325ad5eb6846/kube-rbac-proxy/0.log"
Jul 15 05:51:14.013676 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:14.013652    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n88cb_8c760c2f-4a4b-4d8d-990f-325ad5eb6846/init-textfile/0.log"
Jul 15 05:51:14.147425 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:14.147398    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddd0578b-11d7-4599-babd-353350e5d10c/prometheus/0.log"
Jul 15 05:51:14.171666 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:14.171563    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddd0578b-11d7-4599-babd-353350e5d10c/config-reloader/0.log"
Jul 15 05:51:14.193191 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:14.193152    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddd0578b-11d7-4599-babd-353350e5d10c/thanos-sidecar/0.log"
Jul 15 05:51:14.213257 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:14.213223    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddd0578b-11d7-4599-babd-353350e5d10c/kube-rbac-proxy-web/0.log"
Jul 15 05:51:14.234402 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:14.234303    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddd0578b-11d7-4599-babd-353350e5d10c/kube-rbac-proxy/0.log"
Jul 15 05:51:14.259511 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:14.259485    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddd0578b-11d7-4599-babd-353350e5d10c/kube-rbac-proxy-thanos/0.log"
Jul 15 05:51:14.276787 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:14.276759    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ddd0578b-11d7-4599-babd-353350e5d10c/init-config-reloader/0.log"
Jul 15 05:51:17.818883 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:17.818834    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"]
Jul 15 05:51:17.824544 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:17.824513    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:17.828688 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:17.828660    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"]
Jul 15 05:51:17.961976 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:17.961929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bd5t\" (UniqueName: \"kubernetes.io/projected/8cb83902-cbba-4800-8962-10103e0eade8-kube-api-access-4bd5t\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:17.962175 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:17.962020    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cb83902-cbba-4800-8962-10103e0eade8-lib-modules\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:17.962175 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:17.962043    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8cb83902-cbba-4800-8962-10103e0eade8-podres\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:17.962175 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:17.962083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cb83902-cbba-4800-8962-10103e0eade8-sys\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:17.962175 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:17.962108    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8cb83902-cbba-4800-8962-10103e0eade8-proc\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:18.063501 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:18.063465    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cb83902-cbba-4800-8962-10103e0eade8-lib-modules\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:18.063710 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:18.063606    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8cb83902-cbba-4800-8962-10103e0eade8-podres\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:18.063710 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:18.063625    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cb83902-cbba-4800-8962-10103e0eade8-lib-modules\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:18.063710 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:18.063662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cb83902-cbba-4800-8962-10103e0eade8-sys\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:18.063710 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:18.063693    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8cb83902-cbba-4800-8962-10103e0eade8-proc\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:18.063897 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:18.063722    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8cb83902-cbba-4800-8962-10103e0eade8-podres\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:18.063897 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:18.063781    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cb83902-cbba-4800-8962-10103e0eade8-sys\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:18.063897 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:18.063791    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8cb83902-cbba-4800-8962-10103e0eade8-proc\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:18.063897 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:18.063835    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bd5t\" (UniqueName: \"kubernetes.io/projected/8cb83902-cbba-4800-8962-10103e0eade8-kube-api-access-4bd5t\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:18.071855 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:18.071780    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bd5t\" (UniqueName: \"kubernetes.io/projected/8cb83902-cbba-4800-8962-10103e0eade8-kube-api-access-4bd5t\") pod \"perf-node-gather-daemonset-6ghz8\" (UID: \"8cb83902-cbba-4800-8962-10103e0eade8\") " pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:18.139426 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:18.139384    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:18.320060 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:18.319980    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"]
Jul 15 05:51:18.330192 ip-10-0-128-231 kubenswrapper[2562]: W0715 05:51:18.330155    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8cb83902_cbba_4800_8962_10103e0eade8.slice/crio-2a69fbcdfb5bd6b09859b7f5279c805f02c7118f8de4c7104a3d2e3b77f9334e WatchSource:0}: Error finding container 2a69fbcdfb5bd6b09859b7f5279c805f02c7118f8de4c7104a3d2e3b77f9334e: Status 404 returned error can't find the container with id 2a69fbcdfb5bd6b09859b7f5279c805f02c7118f8de4c7104a3d2e3b77f9334e
Jul 15 05:51:18.342429 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:18.342086    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8" event={"ID":"8cb83902-cbba-4800-8962-10103e0eade8","Type":"ContainerStarted","Data":"2a69fbcdfb5bd6b09859b7f5279c805f02c7118f8de4c7104a3d2e3b77f9334e"}
Jul 15 05:51:19.241675 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:19.241637    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bhmbw_be469964-3786-4205-a37b-5a0198eed10d/dns/0.log"
Jul 15 05:51:19.261942 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:19.261915    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bhmbw_be469964-3786-4205-a37b-5a0198eed10d/kube-rbac-proxy/0.log"
Jul 15 05:51:19.283515 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:19.283491    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4tjhh_b13f25c6-5faa-4d10-94e9-1174def82690/dns-node-resolver/0.log"
Jul 15 05:51:19.348920 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:19.348872    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8" event={"ID":"8cb83902-cbba-4800-8962-10103e0eade8","Type":"ContainerStarted","Data":"b882200a345642ef9782f292d249fd7e8f22e8887694f4c3cce70152e4ba99da"}
Jul 15 05:51:19.349777 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:19.349746    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:19.367687 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:19.367632    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8" podStartSLOduration=2.367610485 podStartE2EDuration="2.367610485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 05:51:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 05:51:19.364621747 +0000 UTC m=+3161.968174352" watchObservedRunningTime="2026-07-15 05:51:19.367610485 +0000 UTC m=+3161.971163095"
Jul 15 05:51:19.843490 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:19.843458    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-56894f6479-5p4v4_9c5193cb-c715-4860-a390-3e3f228be090/registry/0.log"
Jul 15 05:51:19.864260 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:19.864223    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6tc82_e3d94ce3-5595-45ad-90e8-b5da6d480cc5/node-ca/0.log"
Jul 15 05:51:21.030591 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:21.030559    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57bf8bf567-pzr2h_0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61/kube-auth-proxy/0.log"
Jul 15 05:51:21.927430 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:21.927389    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p7b4f_c99848fc-5c0a-4482-8535-10bdd21098d2/serve-healthcheck-canary/0.log"
Jul 15 05:51:22.369565 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:22.369511    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-zkpg2_e2f8ae05-0862-4e90-bde9-1f743802015a/insights-operator/0.log"
Jul 15 05:51:22.370468 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:22.370448    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-zkpg2_e2f8ae05-0862-4e90-bde9-1f743802015a/insights-operator/1.log"
Jul 15 05:51:22.392896 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:22.392872    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2dqv5_a222e79e-cdf0-4e4f-aa18-3a30cbede232/kube-rbac-proxy/0.log"
Jul 15 05:51:22.413443 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:22.413412    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2dqv5_a222e79e-cdf0-4e4f-aa18-3a30cbede232/exporter/0.log"
Jul 15 05:51:22.435607 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:22.435582    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2dqv5_a222e79e-cdf0-4e4f-aa18-3a30cbede232/extractor/0.log"
Jul 15 05:51:25.192960 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:25.192908    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749c75689b-sqgqr_1780b906-079c-414c-aa53-aea05c06bbbf/manager/0.log"
Jul 15 05:51:25.211941 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:25.211915    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749c75689b-sqgqr_1780b906-079c-414c-aa53-aea05c06bbbf/copy-manifests/0.log"
Jul 15 05:51:26.336390 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:26.336338    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8fd856859-bmcmr_ca6e8cec-ddb6-44a0-91de-86bad50c8d6a/manager/0.log"
Jul 15 05:51:26.371015 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:26.370987    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bw8rh/perf-node-gather-daemonset-6ghz8"
Jul 15 05:51:27.810746 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:27.810658    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57bf8bf567-pzr2h_0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61/kube-auth-proxy/0.log"
Jul 15 05:51:29.594838 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:29.594808    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57bf8bf567-pzr2h_0b421add-ecf3-4b58-a6a0-cdfe0fb9cd61/kube-auth-proxy/0.log"
Jul 15 05:51:31.685473 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:31.685433    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-9xj54_249d0866-7a1b-44f2-a7b8-68788fb7b935/migrator/0.log"
Jul 15 05:51:31.704705 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:31.704674    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-9xj54_249d0866-7a1b-44f2-a7b8-68788fb7b935/graceful-termination/0.log"
Jul 15 05:51:33.049323 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:33.049290    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6rksr_674d12b8-5912-4f32-b965-1a498763f81c/kube-multus/0.log"
Jul 15 05:51:33.218463 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:33.218435    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b46gk_24d3a875-2a2f-47a9-9842-c12d04ad9fb9/kube-multus-additional-cni-plugins/0.log"
Jul 15 05:51:33.237741 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:33.237712    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b46gk_24d3a875-2a2f-47a9-9842-c12d04ad9fb9/egress-router-binary-copy/0.log"
Jul 15 05:51:33.256613 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:33.256586    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b46gk_24d3a875-2a2f-47a9-9842-c12d04ad9fb9/cni-plugins/0.log"
Jul 15 05:51:33.277546 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:33.277513    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b46gk_24d3a875-2a2f-47a9-9842-c12d04ad9fb9/bond-cni-plugin/0.log"
Jul 15 05:51:33.299627 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:33.299567    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b46gk_24d3a875-2a2f-47a9-9842-c12d04ad9fb9/routeoverride-cni/0.log"
Jul 15 05:51:33.320775 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:33.320751    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b46gk_24d3a875-2a2f-47a9-9842-c12d04ad9fb9/whereabouts-cni-bincopy/0.log"
Jul 15 05:51:33.341001 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:33.340977    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b46gk_24d3a875-2a2f-47a9-9842-c12d04ad9fb9/whereabouts-cni/0.log"
Jul 15 05:51:33.539952 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:33.539918    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9vxv5_30b62c6b-b454-403b-b1be-4fd921f4a5e4/network-metrics-daemon/0.log"
Jul 15 05:51:33.558114 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:33.558047    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9vxv5_30b62c6b-b454-403b-b1be-4fd921f4a5e4/kube-rbac-proxy/0.log"
Jul 15 05:51:34.430321 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:34.430274    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-controller/0.log"
Jul 15 05:51:34.488685 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:34.488650    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/0.log"
Jul 15 05:51:34.488858 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:34.488691    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovn-acl-logging/1.log"
Jul 15 05:51:34.543084 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:34.543052    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/kube-rbac-proxy-node/0.log"
Jul 15 05:51:34.584563 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:34.584536    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/kube-rbac-proxy-ovn-metrics/0.log"
Jul 15 05:51:34.601225 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:34.601199    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/northd/0.log"
Jul 15 05:51:34.621744 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:34.621713    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/nbdb/0.log"
Jul 15 05:51:34.643070 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:34.643043    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/sbdb/0.log"
Jul 15 05:51:34.702850 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:34.702816    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djb4c_e1aacd1d-1083-4b6d-8eb6-064cb6be4430/ovnkube-controller/0.log"
Jul 15 05:51:36.338423 ip-10-0-128-231 kubenswrapper[2562]: I0715 05:51:36.338390    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-v8c4q_8bd2c88a-1300-41b1-984f-fc4d17ff45f6/network-check-target-container/0.log"